-- Logs begin at Wed 2022-06-15 23:40:13 UTC, end at Thu 2022-06-16 00:58:40 UTC. -- Jun 16 00:15:25.601481 dv-cinder-803823-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 00:15:25.611564 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 00:15:25 2022] *** Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: nodename: dv-cinder-803823-11-iscsi Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: machine: x86_64 Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: clock source: unix Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: pcre jit disabled Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: detected number of CPU cores: 4 Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: current working directory: / Jun 16 00:15:25.612625 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: detected binary path: /usr/bin/uwsgi-core Jun 16 00:15:25.613199 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: your processes number limit is 47680 Jun 16 00:15:25.613199 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: your memory page size is 4096 bytes Jun 16 00:15:25.613199 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: detected max file descriptor number: 2048 Jun 16 00:15:25.613199 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: lock engine: pthread robust mutexes Jun 16 00:15:25.613199 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: thunder lock: enabled Jun 16 00:15:25.613199 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 00:15:25.613199 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 00:15:25.656732 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: Python main interpreter initialized at 0x55ba3fa5c9c0 Jun 16 00:15:25.656732 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: python threads support enabled Jun 16 00:15:25.656732 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: your server socket listen backlog is limited to 100 connections Jun 16 00:15:25.656732 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: your mercy for graceful operations on workers is 90 seconds Jun 16 00:15:25.657126 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 00:15:25.657200 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: *** Operational MODE: preforking *** Jun 16 00:15:25.657436 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: *** uWSGI is running in multiple interpreter mode *** Jun 16 00:15:25.657436 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: spawned uWSGI master process (pid: 86670) Jun 16 00:15:25.657643 dv-cinder-803823-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 00:15:25.660155 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: spawned uWSGI worker 1 (pid: 86685, cores: 1) Jun 16 00:15:25.660155 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: spawned uWSGI worker 2 (pid: 86686, cores: 1) Jun 16 00:15:25.660155 dv-cinder-803823-11-iscsi devstack@keystone.service[86670]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 00:15:31.149519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 00:15:31.149519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: from cryptography.utils import int_from_bytes Jun 16 00:15:31.150298 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 00:15:31.150298 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: from cryptography.utils import int_from_bytes Jun 16 00:15:31.150426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 00:15:31.150426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: from cryptography.utils import int_from_bytes Jun 16 00:15:31.150426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 00:15:31.150426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: from cryptography.utils import int_from_bytes Jun 16 00:15:31.200490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 00:15:31.200748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 00:15:31.200902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] command line args: [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 00:15:31.201176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 00:15:31.201355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ================================================================================ {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 00:15:31.201610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] admin_token = **** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.201855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] config_dir = [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.202043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.202226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] config_source = [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.202406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] debug = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.202606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.202850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 00:15:31.202973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.203057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 00:15:31.203155 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.203260 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] command line args: [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 00:15:31.203399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.203510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 00:15:31.203611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ================================================================================ {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 00:15:31.203712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.203861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] admin_token = **** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.203976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] list_limit = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] log_config_append = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204177 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] config_dir = [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204380 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] log_dir = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] config_source = [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] log_file = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] debug = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.204939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.205921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] list_limit = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] log_config_append = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] log_dir = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] log_file = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.206940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.207951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.208085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.208322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.208441 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.208586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.208684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] publish_errors = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.208826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.208903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.209063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.209187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.209312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.209415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.209526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.209644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.209756 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.209867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.210030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] standard_threads = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.210147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.210260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.210371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.210515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.210656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.210771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] use_journal = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.210889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] publish_errors = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] use_json = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] use_stderr = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] use_syslog = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.211942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.212061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.212180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.212295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] standard_threads = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.212407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.212546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.212673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.212788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.212901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.213013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.213013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] use_journal = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.213958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.214076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] use_json = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.214189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.214309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] use_stderr = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.214309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] use_syslog = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.214522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:15:31.214757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.214876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.214991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.215109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.215233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.215353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.215467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.215582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.215701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.215814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.215933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.216053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.216166 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.216284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.216403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.216553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.216683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.216804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.216917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.217954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.218078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.218192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.218310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.218423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.218537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.218656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.218779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.218900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.219013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.219125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.219237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.219355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.219542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.219659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.219782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.219897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.220953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.221961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222852 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.222972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.223087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.223211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.223356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.223486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.223607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.223721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.223835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.223948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.224071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.224193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.224298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.224410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.224545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.224661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.224764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.224892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.225936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.226977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.227100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.227220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.227342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.227469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.227584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.227698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.227812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.227933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.228048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.228168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.228282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.228396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.228509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.228681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.228831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.228948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.229979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.230105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.230227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.230341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.230460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.230574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.230687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.230877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.230994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.231108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.231222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.231335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.231448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.231573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.231690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.231802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.231916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.232028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.232148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.232275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.232402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.232537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.232654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.232767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.232890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233243 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.233934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234166 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.234974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.235676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.236297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.236297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.236557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.236675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.236799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.236915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.237034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.237149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.237270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.237270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.237486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.237598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] auth.external = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.237797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.238106 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.238349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.238473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.238589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] auth.password = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.238702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.238911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] auth.token = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.239029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.239158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.239301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.239432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.239554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.239714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] auth.external = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.239831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.239946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.240058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.240181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.240297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.240412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.240545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.240662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.240788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] auth.password = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.240905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] credential.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241019 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] auth.token = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241133 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.241942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.242056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.242178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.242294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.242414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.242529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.242656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.242771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] credential.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.242893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.243941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.244054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.244167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.244288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.244434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.244601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.244719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.244834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] federation.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.244948 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.245137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.245255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.245370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.245484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.245607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.245723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.245845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.245964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] federation.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.246998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.247113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.247234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.247350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.247464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.247578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.247691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.247805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.247929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.248044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.248159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.248273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.248427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.248570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.248699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.248815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.248931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.249985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.250107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.250223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.250344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.250459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.250580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.250700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.250824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.250940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.251889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.252111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.252226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.252339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.252452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.252587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.252702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.252823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.252937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.253969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.254082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.254196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.254310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.254423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.254542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.254657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.254769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.254891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.255923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.256037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.256150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.256273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.256387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.256500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.256655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.256770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.256897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257481 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.257933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.258942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.259987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.260610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.260725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.260725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.260951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.261971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.262991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263109 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.263941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264240 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.264993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.265108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.265222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.265336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.265449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.265572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.265689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.265802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.265915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.266032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.266224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.266458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.266708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.266953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.267191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.267309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.267422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.267543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.267658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.267757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.267891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.267997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.268127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.268308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.268409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.268506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.268631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] resource.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.268729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.268858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.268976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.269090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.269189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.269333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.269511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.269631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.269755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.269871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.269985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.270098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.270211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.270325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.270446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.270562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.270675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.270788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.270901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.271015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.271137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] role.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.271252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.271365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] role.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.271479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] resource.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.271592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.271705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.271890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.272140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.272388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.272663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.272913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.273162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.273413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.273647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.273787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.273905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.274069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.274187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.274312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.274428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.274583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.274701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.274853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.275008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.275160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.275262 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.275360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.275486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.275591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.276284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.276402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.276560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.276668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] role.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.276816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.276937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] role.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.277975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.278073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.278204 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.278310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.278438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.278540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.278676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.278796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.278931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279173 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.279985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.280100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] token.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.280214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.280329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.280561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.280693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.280794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.280890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.281976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.282099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.282215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.282330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.282443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.282558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.282672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.282795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.282910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.283902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.284078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.284197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.284313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.284428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.connection = **** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.284577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.284705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.284821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.284936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] token.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.285987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.286976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.287100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.287216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.287331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.287446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.287561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.287675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.287800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.287919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.288034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.288148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.288263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.288377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.288500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.288668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.288668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 00:15:31.288887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.289002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.289235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.connection = **** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.289482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.289733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.289973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.290216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.290453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.290691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.290928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.291166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.291439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.291679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.291928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.292168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.292417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.292680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.292932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.293169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.293412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.293648 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:15:31.293820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 00:15:31.404736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.404908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.405106 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.405499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.405736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.405965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.413896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.414479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.441975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.442205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.442310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.442542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.496792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.497055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.545280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.545429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.545517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.545599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.612001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.612233 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.612347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.612458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.612563 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.612711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.612795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.612954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.613040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.613147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.613228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.613361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.637184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.637330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.637415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.637492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.637635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.637719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.637861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.637944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.638095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.638175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.638281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.638374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.638495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.638576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.649557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.649756 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.662193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.662398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.662495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.662609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.662691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.662827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.662920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.663043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.663126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.663255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.663337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.663468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.663550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.663677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:15:31.680883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.681091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 00:15:31.681231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.681427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.681510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 00:15:31.681630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 00:15:31.681762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.682488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 00:15:31.682586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.682853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.682933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.683115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.683195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.683372 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.683453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.683674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.694171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.694444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.694525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.694713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.694797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.694995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.695081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.695299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.703649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.703969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.707072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.707354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.707434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.707634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.707716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.707915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.707996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.708196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.708278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.708456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.708559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.708743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.708828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.709013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.709106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.709280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.709359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.709546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.709623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.709808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.709888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.710070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.736449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.736763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.738099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.738362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.738446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.738675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.743690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.743978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.744070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.744255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.744335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.745081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.752213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.752515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.752656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.752812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.757014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.757300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.757381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.757582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.757664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.757860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.757940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.758130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.758218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.758401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.758482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.758667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.758747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.758934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.759014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.759202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.778176 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.778420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.778503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.778705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.782932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.783218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.786581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.786862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.787026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.787160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.791967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.792278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.795637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.795876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.799578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.799876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.799957 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.800161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.800255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.800445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.800547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.800748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.800830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.801026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.801109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.801308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.820616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.820900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.820982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.821166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.821250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.821435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.821525 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.821700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.829933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.830194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.831648 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.831902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.831996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.832177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.834989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.835270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.835349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.835552 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.836848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.837125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.837206 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.837413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.837495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.837699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.850740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.850995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.851078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.851275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.851370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.851551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.851632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.851828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.851911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.852098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.852182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.852368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.852449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.852666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.871511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.871762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.874830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.875068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.878182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.878393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.879790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.880017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.881602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.881847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.881929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.882127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.882209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.882409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.889555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.889743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.891168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.891376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.894426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.894636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.894719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.894927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.895031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.895214 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.895297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.895494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.904682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.904881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.904963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.905167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.905249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.905450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.913151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.913347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:15:31.913429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.913626 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.913709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.913908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.913988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.914181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.914263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.914450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.914539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.914723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.914803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.914986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.915067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.915262 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.915343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.915543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.915624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.915885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.915965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.916156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.916763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.916970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.917051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.917234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.917327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.917494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:15:31.955570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ba3fa5c9c0 pid: 86686 (default app) Jun 16 00:15:31.955864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ba3fa5c9c0 pid: 86685 (default app) Jun 16 00:15:31.958956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45bd6-cc6c-4a63-a5be-e8d257ee83a0 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.959251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45bd6-cc6c-4a63-a5be-e8d257ee83a0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.959532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45bd6-cc6c-4a63-a5be-e8d257ee83a0 None None] PATH_INFO: `/v3` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.963735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:40.410998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e075f60-6d12-4e05-bc06-05bf6b8b2269 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:40.411510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e075f60-6d12-4e05-bc06-05bf6b8b2269 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:40.411616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e075f60-6d12-4e05-bc06-05bf6b8b2269 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:40.415921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:40.421441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:40.421821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:40.422244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:40.490645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: INFO dbcounter [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] Registered counter for database keystone Jun 16 00:15:40.511228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG oslo_db.sqlalchemy.engines [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86686) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 00:15:40.517398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writer thread running {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 00:15:40.691734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG passlib.handlers.bcrypt [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86686) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 00:15:40.692092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG passlib.handlers.bcrypt [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86686) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 00:15:40.746447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:40.748050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:40.874500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 00:15:41.020369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e075f60-6d12-4e05-bc06-05bf6b8b2269 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:41.090627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: INFO dbcounter [None req-7e075f60-6d12-4e05-bc06-05bf6b8b2269 None None] Registered counter for database keystone Jun 16 00:15:41.109839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG oslo_db.sqlalchemy.engines [None req-7e075f60-6d12-4e05-bc06-05bf6b8b2269 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86685) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 00:15:41.115955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writer thread running {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 00:15:41.236148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e075f60-6d12-4e05-bc06-05bf6b8b2269 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:41.240474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828e3a-1917-459e-98f8-76985350ed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:41.241176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828e3a-1917-459e-98f8-76985350ed5d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:41.241468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828e3a-1917-459e-98f8-76985350ed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:41.241753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828e3a-1917-459e-98f8-76985350ed5d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:41.250122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828e3a-1917-459e-98f8-76985350ed5d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:41.302130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e1828e3a-1917-459e-98f8-76985350ed5d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 00:15:41.302408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828e3a-1917-459e-98f8-76985350ed5d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:41.312154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Thu Jun 16 00:15:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:15:41.318346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:41.334644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39311ddf-31a3-4f27-8c00-77085ca6010b None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:41.339710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae7b7c-d8b7-4a68-89dc-1e83143ce931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:41.340886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae7b7c-d8b7-4a68-89dc-1e83143ce931 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:41.341175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae7b7c-d8b7-4a68-89dc-1e83143ce931 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:41.341456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae7b7c-d8b7-4a68-89dc-1e83143ce931 admin admin] PATH_INFO: `/v3/projects/26e616646e114f6f885b0d1df925801d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:41.342974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae7b7c-d8b7-4a68-89dc-1e83143ce931 admin admin] RBAC: Authorizing `identity:get_project(project_id=26e616646e114f6f885b0d1df925801d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:41.393050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c5ae7b7c-d8b7-4a68-89dc-1e83143ce931 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 00:15:41.393396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae7b7c-d8b7-4a68-89dc-1e83143ce931 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:41.398021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Jun 16 00:15:41 2022] GET /identity/v3/projects/26e616646e114f6f885b0d1df925801d => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:43.484470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d12ff-c71a-4675-be80-ddb0c278f715 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:43.485064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d12ff-c71a-4675-be80-ddb0c278f715 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:43.485064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d12ff-c71a-4675-be80-ddb0c278f715 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:43.489179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:43.495079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9ab0-998c-4781-a934-a2490f80770e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:43.495492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9ab0-998c-4781-a934-a2490f80770e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:43.495631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9ab0-998c-4781-a934-a2490f80770e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:43.514954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d4cf9ab0-998c-4781-a934-a2490f80770e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:43.516470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d4cf9ab0-998c-4781-a934-a2490f80770e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:43.524258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 00:15:43.547482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d12ff-c71a-4675-be80-ddb0c278f715 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:43.559870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d12ff-c71a-4675-be80-ddb0c278f715 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:43.564069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e65c5-edef-4a82-b116-fc188f5039b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:43.564730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e65c5-edef-4a82-b116-fc188f5039b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:43.565020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e65c5-edef-4a82-b116-fc188f5039b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:43.565300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e65c5-edef-4a82-b116-fc188f5039b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:43.573559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e65c5-edef-4a82-b116-fc188f5039b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:43.575324 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e65c5-edef-4a82-b116-fc188f5039b8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:43.584122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Thu Jun 16 00:15:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:15:43.589419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9ab0-998c-4781-a934-a2490f80770e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:43.598391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9ab0-998c-4781-a934-a2490f80770e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:43.602723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab6fb8-47a0-4dc1-9ba5-b647036b4ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:43.603584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab6fb8-47a0-4dc1-9ba5-b647036b4ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:43.603868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab6fb8-47a0-4dc1-9ba5-b647036b4ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:43.604142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab6fb8-47a0-4dc1-9ba5-b647036b4ea6 admin admin] PATH_INFO: `/v3/users/1aa1627cba2640fdb90f9d237f108268` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:43.605565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab6fb8-47a0-4dc1-9ba5-b647036b4ea6 admin admin] RBAC: Authorizing `identity:get_user(user_id=1aa1627cba2640fdb90f9d237f108268)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:43.607381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab6fb8-47a0-4dc1-9ba5-b647036b4ea6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:43.608973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Thu Jun 16 00:15:43 2022] GET /identity/v3/users/1aa1627cba2640fdb90f9d237f108268 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:45.796619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a272c08-31a6-4df5-90f3-698ebe60f30c None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:45.797386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a272c08-31a6-4df5-90f3-698ebe60f30c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:45.797490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a272c08-31a6-4df5-90f3-698ebe60f30c None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:45.798787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:45.803933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd6ad-8f8f-43bd-b899-0f71ce5933f7 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:45.804353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd6ad-8f8f-43bd-b899-0f71ce5933f7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:45.804491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfd6ad-8f8f-43bd-b899-0f71ce5933f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:45.813123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b023d0-e4b1-424a-81a6-40f02121c45f None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:45.813418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b023d0-e4b1-424a-81a6-40f02121c45f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:45.813701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b023d0-e4b1-424a-81a6-40f02121c45f None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:45.815002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:45.820098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:45.820392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:45.820720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:45.820904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-22cfd6ad-8f8f-43bd-b899-0f71ce5933f7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:45.822085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-22cfd6ad-8f8f-43bd-b899-0f71ce5933f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:45.860381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG passlib.handlers.bcrypt [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86685) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 00:15:45.860761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG passlib.handlers.bcrypt [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86685) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 00:15:45.901803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:15:45.907182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:45.908789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:45.925638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfd6ad-8f8f-43bd-b899-0f71ce5933f7 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:45.948898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.007243 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:15:46.026624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfd6ad-8f8f-43bd-b899-0f71ce5933f7 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.029206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.060353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dff898-906d-481e-b6b5-ea21269731be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.061174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dff898-906d-481e-b6b5-ea21269731be None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.061468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dff898-906d-481e-b6b5-ea21269731be None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.061750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dff898-906d-481e-b6b5-ea21269731be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.073281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dff898-906d-481e-b6b5-ea21269731be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-e6dff898-906d-481e-b6b5-ea21269731be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:46.084254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:46.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:15:46.086080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:15:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:46.091886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dff898-906d-481e-b6b5-ea21269731be None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.131933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84398163-1821-411d-a45b-b13eaba8c46b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.134128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.168138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017c132-b8fe-41aa-9019-ca3e6118b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.169166 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017c132-b8fe-41aa-9019-ca3e6118b91a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.169457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017c132-b8fe-41aa-9019-ca3e6118b91a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.169739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017c132-b8fe-41aa-9019-ca3e6118b91a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.171507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.189569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dff898-906d-481e-b6b5-ea21269731be None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.223368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.224036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.224323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.224633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.225390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:46.225701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.227094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.236086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017c132-b8fe-41aa-9019-ca3e6118b91a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.237616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017c132-b8fe-41aa-9019-ca3e6118b91a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.238629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:46.244060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.332770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:15:46.338776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017c132-b8fe-41aa-9019-ca3e6118b91a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.341404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.343170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2181a49-b936-490d-8155-4dfd5f54dedf None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.376636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17762cb-917f-4989-a6ff-3c0bc7c7d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.377509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17762cb-917f-4989-a6ff-3c0bc7c7d322 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.377904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17762cb-917f-4989-a6ff-3c0bc7c7d322 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.378082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17762cb-917f-4989-a6ff-3c0bc7c7d322 None admin] PATH_INFO: `/v3/users/1aa1627cba2640fdb90f9d237f108268` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.379299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17762cb-917f-4989-a6ff-3c0bc7c7d322 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa1627cba2640fdb90f9d237f108268)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.380793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17762cb-917f-4989-a6ff-3c0bc7c7d322 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.382466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/users/1aa1627cba2640fdb90f9d237f108268 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:46.388256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17762cb-917f-4989-a6ff-3c0bc7c7d322 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.403249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.437190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017c132-b8fe-41aa-9019-ca3e6118b91a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.472490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.473311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.473596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.473877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.486628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17762cb-917f-4989-a6ff-3c0bc7c7d322 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.491628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.493126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:46.514538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:46.515433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:46.516245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:46.516245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:46.516245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:46.516245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:15:46.516245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:46.520768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a5db2-bbad-47d7-a727-3cf8f339aded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.521370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.521565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a5db2-bbad-47d7-a727-3cf8f339aded None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.521855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a5db2-bbad-47d7-a727-3cf8f339aded None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.522142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a5db2-bbad-47d7-a727-3cf8f339aded None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.523321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a5db2-bbad-47d7-a727-3cf8f339aded None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.524790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a5db2-bbad-47d7-a727-3cf8f339aded None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.526433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:46.531932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a5db2-bbad-47d7-a727-3cf8f339aded None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.577113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.615594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.624291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d6661-1bd0-47dc-a359-0fe04edf949a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.635681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a5db2-bbad-47d7-a727-3cf8f339aded None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.657556 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ddaa0-32be-4a35-bc5a-c52097396292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.658213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ddaa0-32be-4a35-bc5a-c52097396292 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.658494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ddaa0-32be-4a35-bc5a-c52097396292 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.658772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ddaa0-32be-4a35-bc5a-c52097396292 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.659489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ddaa0-32be-4a35-bc5a-c52097396292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:46.659798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ddaa0-32be-4a35-bc5a-c52097396292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.661203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ddaa0-32be-4a35-bc5a-c52097396292 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.668240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb30c1-5452-491c-a3b9-c1a2084b816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.668929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb30c1-5452-491c-a3b9-c1a2084b816b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.669218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb30c1-5452-491c-a3b9-c1a2084b816b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.669494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb30c1-5452-491c-a3b9-c1a2084b816b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.670339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb30c1-5452-491c-a3b9-c1a2084b816b None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.domain.id=default, user.id=1aa1627cba2640fdb90f9d237f108268) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:46.670655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb30c1-5452-491c-a3b9-c1a2084b816b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.672354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb30c1-5452-491c-a3b9-c1a2084b816b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.678322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:46.678947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/role_assignments?user.id=1aa1627cba2640fdb90f9d237f108268&scope.domain.id=default&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:49.027187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb25c5-2b12-454c-a0bf-ac4ad6bb6c9c None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.027732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb25c5-2b12-454c-a0bf-ac4ad6bb6c9c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.027732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb25c5-2b12-454c-a0bf-ac4ad6bb6c9c None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.029140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:49.036902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085e3ca-3763-4422-9964-0e4c3e027241 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.037796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085e3ca-3763-4422-9964-0e4c3e027241 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.038301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085e3ca-3763-4422-9964-0e4c3e027241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.061071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cd750-7220-4b04-a05d-d6b38b92465a None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.061670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cd750-7220-4b04-a05d-d6b38b92465a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.062184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cd750-7220-4b04-a05d-d6b38b92465a None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.063755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 12/23] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:49.073085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4085e3ca-3763-4422-9964-0e4c3e027241 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:49.074890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4085e3ca-3763-4422-9964-0e4c3e027241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:49.081636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae972f9-14ee-420d-ac3b-a165d3033e34 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.082171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae972f9-14ee-420d-ac3b-a165d3033e34 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.082653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae972f9-14ee-420d-ac3b-a165d3033e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.113082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-fae972f9-14ee-420d-ac3b-a165d3033e34 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:49.114622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fae972f9-14ee-420d-ac3b-a165d3033e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:49.248552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 12/24] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:15:49.269237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:15:49.286675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085e3ca-3763-4422-9964-0e4c3e027241 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:49.303321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae972f9-14ee-420d-ac3b-a165d3033e34 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:49.381722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:49.447763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae972f9-14ee-420d-ac3b-a165d3033e34 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:49.489319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:49.504026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d797cd-2a41-4329-b39c-49b594510c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:49.504882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d797cd-2a41-4329-b39c-49b594510c05 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.505174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d797cd-2a41-4329-b39c-49b594510c05 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.505460 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d797cd-2a41-4329-b39c-49b594510c05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.521759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d797cd-2a41-4329-b39c-49b594510c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-64d797cd-2a41-4329-b39c-49b594510c05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:49.532374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:49.533487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:49.534458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:15:49.536548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 14/26] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:15:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:49.543321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d797cd-2a41-4329-b39c-49b594510c05 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:49.555354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085e3ca-3763-4422-9964-0e4c3e027241 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:49.677381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:49.678211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.678504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.678788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.705679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:49.707197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:49.708299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d797cd-2a41-4329-b39c-49b594510c05 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:49.732921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:49.739696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:49.740537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:49.741360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:49.741360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:49.741360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:49.741360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:15:49.749111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 13/27] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:49.755554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:49.765101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2574bf-3bfa-41fe-88e3-4a38ea98b64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:49.765782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2574bf-3bfa-41fe-88e3-4a38ea98b64f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.766069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2574bf-3bfa-41fe-88e3-4a38ea98b64f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.766349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2574bf-3bfa-41fe-88e3-4a38ea98b64f None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.767112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2574bf-3bfa-41fe-88e3-4a38ea98b64f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:49.767425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2574bf-3bfa-41fe-88e3-4a38ea98b64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:49.768833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2574bf-3bfa-41fe-88e3-4a38ea98b64f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:49.786502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 15/28] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:49.788016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883800e-6460-4b79-b564-878527896d38 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.788547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883800e-6460-4b79-b564-878527896d38 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.789031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883800e-6460-4b79-b564-878527896d38 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.790619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 16/29] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:49.796046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883800e-6460-4b79-b564-878527896d38 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:49.907857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883800e-6460-4b79-b564-878527896d38 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:49.964790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca26465-7ac4-4eb9-b0f9-023ca80c84e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:49.966052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca26465-7ac4-4eb9-b0f9-023ca80c84e5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.966562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca26465-7ac4-4eb9-b0f9-023ca80c84e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.966988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ce201-85ea-47a8-8e1a-39e7db5873c3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:49.967279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca26465-7ac4-4eb9-b0f9-023ca80c84e5 None admin] PATH_INFO: `/v3/users/1aa1627cba2640fdb90f9d237f108268` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.969113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca26465-7ac4-4eb9-b0f9-023ca80c84e5 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa1627cba2640fdb90f9d237f108268)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:49.970861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca26465-7ac4-4eb9-b0f9-023ca80c84e5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:49.972862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 17/30] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:49 2022] GET /identity/v3/users/1aa1627cba2640fdb90f9d237f108268 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:49.974264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7c785-09a4-4267-a6bf-fbf60577da05 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.974783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7c785-09a4-4267-a6bf-fbf60577da05 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.975296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7c785-09a4-4267-a6bf-fbf60577da05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.995445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:50.008754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-edc7c785-09a4-4267-a6bf-fbf60577da05 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:50.010479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-edc7c785-09a4-4267-a6bf-fbf60577da05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:50.014199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:50.014336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.014500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.014835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.015604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:50.015912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:50.017337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:50.048815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 14/31] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:50.049601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:50.072018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:50.151144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 18/32] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:15:50.152090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7c785-09a4-4267-a6bf-fbf60577da05 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:50.171015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4ae6d-6056-4f78-9dab-4bbd9d5cf3df None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:50.207457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf86f8f-b17a-4440-8721-c864691ed512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:50.208259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf86f8f-b17a-4440-8721-c864691ed512 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.208576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf86f8f-b17a-4440-8721-c864691ed512 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.208867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf86f8f-b17a-4440-8721-c864691ed512 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.210072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf86f8f-b17a-4440-8721-c864691ed512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:50.211536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf86f8f-b17a-4440-8721-c864691ed512 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:50.214143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 15/33] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:50.215415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1465384-07e1-417e-ac8b-ca612d21e452 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.215706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1465384-07e1-417e-ac8b-ca612d21e452 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.215990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1465384-07e1-417e-ac8b-ca612d21e452 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.217264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 16/34] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:50.217959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1465384-07e1-417e-ac8b-ca612d21e452 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:50.275925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7c785-09a4-4267-a6bf-fbf60577da05 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:50.277769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:50.311696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961facc-4e00-40de-a7f9-efa5d94496b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:50.312291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961facc-4e00-40de-a7f9-efa5d94496b0 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.312647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961facc-4e00-40de-a7f9-efa5d94496b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.312935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961facc-4e00-40de-a7f9-efa5d94496b0 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.313647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961facc-4e00-40de-a7f9-efa5d94496b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:50.315101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961facc-4e00-40de-a7f9-efa5d94496b0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:50.325610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1465384-07e1-417e-ac8b-ca612d21e452 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:50.361672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae21b9-8524-4253-85e3-7acff09a499c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:50.362349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae21b9-8524-4253-85e3-7acff09a499c None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.362636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae21b9-8524-4253-85e3-7acff09a499c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.362922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae21b9-8524-4253-85e3-7acff09a499c None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.363664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae21b9-8524-4253-85e3-7acff09a499c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:50.365080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae21b9-8524-4253-85e3-7acff09a499c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:50.507139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 19/35] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:50 2022] POST /identity/v3/projects => generated 301 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:50.508713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961facc-4e00-40de-a7f9-efa5d94496b0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:50.533769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:50.657709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 17/36] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jun 16 00:15:50 2022] POST /identity/v3/roles => generated 225 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:50.659378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef90e3-c5fd-4586-bd09-e6d896991161 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.663981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef90e3-c5fd-4586-bd09-e6d896991161 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.663981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef90e3-c5fd-4586-bd09-e6d896991161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.680539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-84ef90e3-c5fd-4586-bd09-e6d896991161 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:50.682907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-84ef90e3-c5fd-4586-bd09-e6d896991161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:50.770263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 18/37] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:15:50.772113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deddcc2-1817-4d9a-817b-0d2ca035b33a None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.772620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deddcc2-1817-4d9a-817b-0d2ca035b33a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.773114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deddcc2-1817-4d9a-817b-0d2ca035b33a None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.774042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961facc-4e00-40de-a7f9-efa5d94496b0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:50.774792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 19/38] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:50.776206 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41eec-4167-4c9b-9e47-0691f33776a6 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.776718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41eec-4167-4c9b-9e47-0691f33776a6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.777175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41eec-4167-4c9b-9e47-0691f33776a6 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.778680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 20/39] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:50.780223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b516181-9730-4d33-8298-a2ccf48c4a0c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.780743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b516181-9730-4d33-8298-a2ccf48c4a0c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.781228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b516181-9730-4d33-8298-a2ccf48c4a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.799776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2b516181-9730-4d33-8298-a2ccf48c4a0c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:50.801316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2b516181-9730-4d33-8298-a2ccf48c4a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:50.815361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f94d53-02f8-4a57-9b3c-07295b930b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:50.816242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f94d53-02f8-4a57-9b3c-07295b930b18 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.816562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f94d53-02f8-4a57-9b3c-07295b930b18 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.816849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f94d53-02f8-4a57-9b3c-07295b930b18 None admin] PATH_INFO: `/v3/domains/default/users/1aa1627cba2640fdb90f9d237f108268/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.819464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f94d53-02f8-4a57-9b3c-07295b930b18 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1aa1627cba2640fdb90f9d237f108268, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:50.821154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f94d53-02f8-4a57-9b3c-07295b930b18 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:50.849080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:50.877647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 21/40] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:15:50.878791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e16df-0e2c-468e-90ec-e2aa80102215 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:50.879083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e16df-0e2c-468e-90ec-e2aa80102215 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:50.879363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e16df-0e2c-468e-90ec-e2aa80102215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:50.896446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6f4e16df-0e2c-468e-90ec-e2aa80102215 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:50.897658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6f4e16df-0e2c-468e-90ec-e2aa80102215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:50.906721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 20/41] 10.222.0.20 () {58 vars in 1361 bytes} [Thu Jun 16 00:15:50 2022] PUT /identity/v3/domains/default/users/1aa1627cba2640fdb90f9d237f108268/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:15:50.907332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f94d53-02f8-4a57-9b3c-07295b930b18 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:50.993057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:15:50.994220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e16df-0e2c-468e-90ec-e2aa80102215 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:51.046833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:51.069676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f94d53-02f8-4a57-9b3c-07295b930b18 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:51.114126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e16df-0e2c-468e-90ec-e2aa80102215 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:51.114715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420de8f-1f7f-470f-a394-ea0f4335678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:51.115402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420de8f-1f7f-470f-a394-ea0f4335678f None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:51.115714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420de8f-1f7f-470f-a394-ea0f4335678f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:51.115988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420de8f-1f7f-470f-a394-ea0f4335678f None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:51.116802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420de8f-1f7f-470f-a394-ea0f4335678f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:51.117379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:51.118201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420de8f-1f7f-470f-a394-ea0f4335678f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:51.148856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345881b5-c6d3-4cef-b6de-e40b58f16c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:51.149653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345881b5-c6d3-4cef-b6de-e40b58f16c82 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:51.149979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345881b5-c6d3-4cef-b6de-e40b58f16c82 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:51.150805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345881b5-c6d3-4cef-b6de-e40b58f16c82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:51.151957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345881b5-c6d3-4cef-b6de-e40b58f16c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:51.153557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345881b5-c6d3-4cef-b6de-e40b58f16c82 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:51.155200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 23/43] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:51.155935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345881b5-c6d3-4cef-b6de-e40b58f16c82 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:51.199122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 21/44] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jun 16 00:15:50 2022] POST /identity/v3/roles => generated 219 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:51.200926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420de8f-1f7f-470f-a394-ea0f4335678f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:51.268008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345881b5-c6d3-4cef-b6de-e40b58f16c82 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:51.309371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1e93b-1362-41a0-ac87-21c68f52f3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:51.310761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1e93b-1362-41a0-ac87-21c68f52f3cf None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:51.311274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1e93b-1362-41a0-ac87-21c68f52f3cf None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:51.311751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1e93b-1362-41a0-ac87-21c68f52f3cf None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:51.312791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1e93b-1362-41a0-ac87-21c68f52f3cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:51.314467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1e93b-1362-41a0-ac87-21c68f52f3cf None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:51.334234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420de8f-1f7f-470f-a394-ea0f4335678f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:51.373575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e12220-9c92-4c1e-9118-d4536d1c7836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:51.374178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12220-9c92-4c1e-9118-d4536d1c7836 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:51.374471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12220-9c92-4c1e-9118-d4536d1c7836 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:51.374755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12220-9c92-4c1e-9118-d4536d1c7836 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:51.375486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e12220-9c92-4c1e-9118-d4536d1c7836 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:51.377004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e12220-9c92-4c1e-9118-d4536d1c7836 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:51.386667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:51.415047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 24/45] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jun 16 00:15:51 2022] POST /identity/v3/roles => generated 223 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:51.496860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:51 2022] POST /identity/v3/projects => generated 312 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:53.050816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1e6a5-68f7-4d76-99fa-875bdbf091d5 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.051329 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1e6a5-68f7-4d76-99fa-875bdbf091d5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.051412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1e6a5-68f7-4d76-99fa-875bdbf091d5 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.052792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:53.058722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c57da-564e-4cb3-9249-7af1959af082 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.059113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c57da-564e-4cb3-9249-7af1959af082 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.059301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c57da-564e-4cb3-9249-7af1959af082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.075988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-5f4c57da-564e-4cb3-9249-7af1959af082 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:53.077527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5f4c57da-564e-4cb3-9249-7af1959af082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:53.146849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:15:53.170059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1e6a5-68f7-4d76-99fa-875bdbf091d5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:53.202111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:53.270469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1e6a5-68f7-4d76-99fa-875bdbf091d5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:53.307091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b38458-1193-4a3f-aea1-a99b1f73db57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:53.307815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b38458-1193-4a3f-aea1-a99b1f73db57 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.308106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b38458-1193-4a3f-aea1-a99b1f73db57 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.308388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b38458-1193-4a3f-aea1-a99b1f73db57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.318770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b38458-1193-4a3f-aea1-a99b1f73db57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-20b38458-1193-4a3f-aea1-a99b1f73db57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:53.326272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:53.327253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:15:53.328037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 26/49] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:15:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:53.333197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c57da-564e-4cb3-9249-7af1959af082 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:53.442581 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c57da-564e-4cb3-9249-7af1959af082 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:53.461982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:53.475731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac5045-d98a-4b66-9388-15a2923208bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:53.476410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac5045-d98a-4b66-9388-15a2923208bb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.476735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac5045-d98a-4b66-9388-15a2923208bb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.477019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac5045-d98a-4b66-9388-15a2923208bb None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.477800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac5045-d98a-4b66-9388-15a2923208bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:53.478113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac5045-d98a-4b66-9388-15a2923208bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:53.481045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac5045-d98a-4b66-9388-15a2923208bb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:53.489855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 24/50] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:53.495504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b38458-1193-4a3f-aea1-a99b1f73db57 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:53.592331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:53.600824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b38458-1193-4a3f-aea1-a99b1f73db57 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:53.627571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ef3cf-0d82-4971-9ba1-cb3d1c503387 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.627869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ef3cf-0d82-4971-9ba1-cb3d1c503387 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.628154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ef3cf-0d82-4971-9ba1-cb3d1c503387 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.629558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 25/51] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:53.635064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17c1d-a206-4d5a-bf4d-50f5f7afe106 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.635305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c54f95-9536-4cb7-b0e0-316bf9056fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:53.635419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17c1d-a206-4d5a-bf4d-50f5f7afe106 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.635636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17c1d-a206-4d5a-bf4d-50f5f7afe106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.636192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54f95-9536-4cb7-b0e0-316bf9056fe3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.636478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54f95-9536-4cb7-b0e0-316bf9056fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.636786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54f95-9536-4cb7-b0e0-316bf9056fe3 None admin] PATH_INFO: `/v3/users/1aa1627cba2640fdb90f9d237f108268` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.637963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c54f95-9536-4cb7-b0e0-316bf9056fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa1627cba2640fdb90f9d237f108268)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:53.639415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c54f95-9536-4cb7-b0e0-316bf9056fe3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:53.641059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 27/52] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:53 2022] GET /identity/v3/users/1aa1627cba2640fdb90f9d237f108268 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:53.646310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c54f95-9536-4cb7-b0e0-316bf9056fe3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:53.653351 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-5ab17c1d-a206-4d5a-bf4d-50f5f7afe106 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:53.657230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5ab17c1d-a206-4d5a-bf4d-50f5f7afe106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:53.727113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 26/53] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:15:53.750341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab17c1d-a206-4d5a-bf4d-50f5f7afe106 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:53.754138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c54f95-9536-4cb7-b0e0-316bf9056fe3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:53.788592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c909f1c-f3f5-4756-aca1-90ac1cfaf552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:53.789390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909f1c-f3f5-4756-aca1-90ac1cfaf552 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.789689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909f1c-f3f5-4756-aca1-90ac1cfaf552 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.790004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909f1c-f3f5-4756-aca1-90ac1cfaf552 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.791198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c909f1c-f3f5-4756-aca1-90ac1cfaf552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:53.792687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c909f1c-f3f5-4756-aca1-90ac1cfaf552 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:53.794331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 28/54] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:53.800427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c909f1c-f3f5-4756-aca1-90ac1cfaf552 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:53.828657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:53.848882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab17c1d-a206-4d5a-bf4d-50f5f7afe106 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:53.861502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:53.882364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac480f-74c7-491f-a18c-6c689bc02caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:53.883165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac480f-74c7-491f-a18c-6c689bc02caa None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.883452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac480f-74c7-491f-a18c-6c689bc02caa None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.883734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac480f-74c7-491f-a18c-6c689bc02caa None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.885494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac480f-74c7-491f-a18c-6c689bc02caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:53.886936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac480f-74c7-491f-a18c-6c689bc02caa None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:53.888606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 27/55] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:53.894121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac480f-74c7-491f-a18c-6c689bc02caa None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:53.899882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c909f1c-f3f5-4756-aca1-90ac1cfaf552 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:53.933318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbac10c-972f-4b94-9a0e-f6e1b4d87793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:53.935065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac10c-972f-4b94-9a0e-f6e1b4d87793 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:53.935065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac10c-972f-4b94-9a0e-f6e1b4d87793 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:53.935065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac10c-972f-4b94-9a0e-f6e1b4d87793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:53.935360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbac10c-972f-4b94-9a0e-f6e1b4d87793 None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.domain.id=default, user.id=1aa1627cba2640fdb90f9d237f108268) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:53.935657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbac10c-972f-4b94-9a0e-f6e1b4d87793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:53.937139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbac10c-972f-4b94-9a0e-f6e1b4d87793 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:53.942900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 29/56] 10.222.0.20 () {56 vars in 1322 bytes} [Thu Jun 16 00:15:53 2022] GET /identity/v3/role_assignments?user.id=1aa1627cba2640fdb90f9d237f108268&scope.domain.id=default&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:53.997161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac480f-74c7-491f-a18c-6c689bc02caa None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:54.033828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb131946-f252-45b1-856c-9061e05610ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:54.034505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb131946-f252-45b1-856c-9061e05610ef None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:54.034791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb131946-f252-45b1-856c-9061e05610ef None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:54.035069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb131946-f252-45b1-856c-9061e05610ef None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:54.035796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb131946-f252-45b1-856c-9061e05610ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:54.037299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb131946-f252-45b1-856c-9061e05610ef None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:54.113194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 28/57] 10.222.0.20 () {60 vars in 1140 bytes} [Thu Jun 16 00:15:53 2022] POST /identity/v3/projects => generated 298 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:56.235911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e932-e0db-47ad-9cac-6ef1c37ba7ba None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:56.236450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e932-e0db-47ad-9cac-6ef1c37ba7ba None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:56.236565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e932-e0db-47ad-9cac-6ef1c37ba7ba None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:56.237885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 30/58] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:56.243008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a0569-94b7-4dfc-b0d5-3accc2486df2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:56.243406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a0569-94b7-4dfc-b0d5-3accc2486df2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:56.243582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a0569-94b7-4dfc-b0d5-3accc2486df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:56.259925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-868a0569-94b7-4dfc-b0d5-3accc2486df2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:56.261163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-868a0569-94b7-4dfc-b0d5-3accc2486df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:56.286291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:56.325761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 29/59] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:15:56.349940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81e932-e0db-47ad-9cac-6ef1c37ba7ba None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:56.440922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:56.450733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81e932-e0db-47ad-9cac-6ef1c37ba7ba None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:56.483964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4d4ec-555b-46de-8757-560d6d6698af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:56.484781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d4ec-555b-46de-8757-560d6d6698af None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:56.485073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d4ec-555b-46de-8757-560d6d6698af None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:56.485354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d4ec-555b-46de-8757-560d6d6698af None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:56.486581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4d4ec-555b-46de-8757-560d6d6698af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:56.488011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4d4ec-555b-46de-8757-560d6d6698af None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:56.489620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 31/60] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:56.495631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a0569-94b7-4dfc-b0d5-3accc2486df2 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:56.596438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a0569-94b7-4dfc-b0d5-3accc2486df2 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:56.629722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97061953-60f6-4f36-8d2d-23b5dd5ee433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:56.630388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97061953-60f6-4f36-8d2d-23b5dd5ee433 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:56.630677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97061953-60f6-4f36-8d2d-23b5dd5ee433 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:56.631018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97061953-60f6-4f36-8d2d-23b5dd5ee433 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:56.631738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97061953-60f6-4f36-8d2d-23b5dd5ee433 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:56.633553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97061953-60f6-4f36-8d2d-23b5dd5ee433 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:56.734327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 30/61] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:15:56 2022] POST /identity/v3/users => generated 275 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:00.191171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7688-ecd3-4013-b05a-ca611e282f50 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.192041 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7688-ecd3-4013-b05a-ca611e282f50 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.192466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7688-ecd3-4013-b05a-ca611e282f50 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.194371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 32/62] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:00.199866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc6062-3b86-484c-9e16-564308d1287b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.200713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc6062-3b86-484c-9e16-564308d1287b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.201139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc6062-3b86-484c-9e16-564308d1287b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.204678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d280e3-9148-4ba3-813d-8bfecd1f114a None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.205198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d280e3-9148-4ba3-813d-8bfecd1f114a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.205693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d280e3-9148-4ba3-813d-8bfecd1f114a None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.207238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 33/63] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:00.216376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ca20a-84c4-43f8-a6a9-653c3fd122c1 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.216930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ca20a-84c4-43f8-a6a9-653c3fd122c1 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.217405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ca20a-84c4-43f8-a6a9-653c3fd122c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.240669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-041ca20a-84c4-43f8-a6a9-653c3fd122c1 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:00.242203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-041ca20a-84c4-43f8-a6a9-653c3fd122c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:00.243370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-63cc6062-3b86-484c-9e16-564308d1287b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:00.244637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-63cc6062-3b86-484c-9e16-564308d1287b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:00.258680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:00.320736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:00.323550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 31/65] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:00.329721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf202f73-bead-4e4e-943f-8110169b70b7 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.330111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf202f73-bead-4e4e-943f-8110169b70b7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.330534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf202f73-bead-4e4e-943f-8110169b70b7 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.332114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 35/66] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:00.341992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029062b7-5fb9-4d60-9610-f76ef8da880b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.342288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029062b7-5fb9-4d60-9610-f76ef8da880b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.342727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029062b7-5fb9-4d60-9610-f76ef8da880b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.357874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf202f73-bead-4e4e-943f-8110169b70b7 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:00.367082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-029062b7-5fb9-4d60-9610-f76ef8da880b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:00.368287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-029062b7-5fb9-4d60-9610-f76ef8da880b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:00.438941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 32/67] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:00.440028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029062b7-5fb9-4d60-9610-f76ef8da880b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:00.475231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:00.486421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf202f73-bead-4e4e-943f-8110169b70b7 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:00.525474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:00.528842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55772465-fffc-4974-9254-3385f8484313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:00.530171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55772465-fffc-4974-9254-3385f8484313 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.530624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55772465-fffc-4974-9254-3385f8484313 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.530915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55772465-fffc-4974-9254-3385f8484313 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.544147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55772465-fffc-4974-9254-3385f8484313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-55772465-fffc-4974-9254-3385f8484313 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:00.552696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:00.553639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:00.560157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:00.560875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029062b7-5fb9-4d60-9610-f76ef8da880b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:00.560955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 36/68] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:00.560955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316428f-effe-458a-8024-bfca0c368674 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.560955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316428f-effe-458a-8024-bfca0c368674 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.560955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316428f-effe-458a-8024-bfca0c368674 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.560955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 37/69] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:00.560955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316428f-effe-458a-8024-bfca0c368674 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:00.607363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e8bc6-8284-4e1e-872e-9df113812a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:00.608174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e8bc6-8284-4e1e-872e-9df113812a7f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.608470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e8bc6-8284-4e1e-872e-9df113812a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.608797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e8bc6-8284-4e1e-872e-9df113812a7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.618831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e8bc6-8284-4e1e-872e-9df113812a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d71e8bc6-8284-4e1e-872e-9df113812a7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:00.626905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:00.627786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:00.628671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 33/70] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:00.629338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e8bc6-8284-4e1e-872e-9df113812a7f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:00.672899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316428f-effe-458a-8024-bfca0c368674 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:00.718397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef24da-1c04-4a87-8baf-13e41fd980ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:00.719194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef24da-1c04-4a87-8baf-13e41fd980ec None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.719480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef24da-1c04-4a87-8baf-13e41fd980ec None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.719764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef24da-1c04-4a87-8baf-13e41fd980ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.730360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef24da-1c04-4a87-8baf-13e41fd980ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:00.734456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e8bc6-8284-4e1e-872e-9df113812a7f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-80ef24da-1c04-4a87-8baf-13e41fd980ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:00.737836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:00.738837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:00.739629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 38/71] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:00.740276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75fe49-e0f3-4721-af26-c8ad1afdec07 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.740646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75fe49-e0f3-4721-af26-c8ad1afdec07 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.740750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75fe49-e0f3-4721-af26-c8ad1afdec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.756135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-db75fe49-e0f3-4721-af26-c8ad1afdec07 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:00.757334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-db75fe49-e0f3-4721-af26-c8ad1afdec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:00.760334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:00.767863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae70746-9412-4e1e-97f3-0a64dd551c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:00.774680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae70746-9412-4e1e-97f3-0a64dd551c25 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.774680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae70746-9412-4e1e-97f3-0a64dd551c25 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.774680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae70746-9412-4e1e-97f3-0a64dd551c25 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.774680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae70746-9412-4e1e-97f3-0a64dd551c25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:00.774680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae70746-9412-4e1e-97f3-0a64dd551c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:00.774680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae70746-9412-4e1e-97f3-0a64dd551c25 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:00.777056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:00.780301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 34/72] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:00.782652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ac2aa-f6a4-4912-b7e2-c45446b66d76 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.783009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ac2aa-f6a4-4912-b7e2-c45446b66d76 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.783296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ac2aa-f6a4-4912-b7e2-c45446b66d76 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.786022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 35/73] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:00.786022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ac2aa-f6a4-4912-b7e2-c45446b66d76 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:00.824851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 39/74] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:00.825703 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75fe49-e0f3-4721-af26-c8ad1afdec07 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:00.889771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ac2aa-f6a4-4912-b7e2-c45446b66d76 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:00.923348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75fe49-e0f3-4721-af26-c8ad1afdec07 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:00.923605 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:00.924281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.924613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.924911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.925674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:00.925989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:00.927367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:00.935784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 36/75] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:00.936614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:00.960822 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df4e2f-d797-4d69-b985-0eda2b19e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:00.961488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df4e2f-d797-4d69-b985-0eda2b19e465 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.961773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df4e2f-d797-4d69-b985-0eda2b19e465 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.962053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df4e2f-d797-4d69-b985-0eda2b19e465 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.963335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df4e2f-d797-4d69-b985-0eda2b19e465 None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:00.963644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df4e2f-d797-4d69-b985-0eda2b19e465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:00.965049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df4e2f-d797-4d69-b985-0eda2b19e465 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:00.973193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 40/76] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:00.974405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b10c34-2da8-4b80-8192-4b09470af6cb None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.974697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b10c34-2da8-4b80-8192-4b09470af6cb None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.974985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b10c34-2da8-4b80-8192-4b09470af6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.980517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:00.992550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-84b10c34-2da8-4b80-8192-4b09470af6cb None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:00.994767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-84b10c34-2da8-4b80-8192-4b09470af6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:01.002548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.039980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ce53c-cc3c-4488-817a-7795abc990bb None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.061026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 41/77] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:01.061851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b10c34-2da8-4b80-8192-4b09470af6cb None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.075797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1dddd-9894-455c-9d88-9036668756be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.076703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1dddd-9894-455c-9d88-9036668756be None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.076993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1dddd-9894-455c-9d88-9036668756be None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.077275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1dddd-9894-455c-9d88-9036668756be None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.092933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1dddd-9894-455c-9d88-9036668756be None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.094400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1dddd-9894-455c-9d88-9036668756be None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.096090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 37/78] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:00 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:01.096889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1dddd-9894-455c-9d88-9036668756be None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.161549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b10c34-2da8-4b80-8192-4b09470af6cb None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.194478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1dddd-9894-455c-9d88-9036668756be None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.194647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.200210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439aea46-9413-4195-9caf-29786aa6243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.201044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439aea46-9413-4195-9caf-29786aa6243c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.201337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439aea46-9413-4195-9caf-29786aa6243c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.201621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439aea46-9413-4195-9caf-29786aa6243c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.211767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439aea46-9413-4195-9caf-29786aa6243c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:01.215008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-439aea46-9413-4195-9caf-29786aa6243c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:01.220009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:01.220967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:01.221849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 42/79] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:01.222901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439aea46-9413-4195-9caf-29786aa6243c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.236795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250472bd-4be8-4c19-901a-0208c3528315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.236795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250472bd-4be8-4c19-901a-0208c3528315 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.236795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250472bd-4be8-4c19-901a-0208c3528315 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.236795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250472bd-4be8-4c19-901a-0208c3528315 None admin] PATH_INFO: `/v3/users/1aa1627cba2640fdb90f9d237f108268` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.237171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250472bd-4be8-4c19-901a-0208c3528315 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa1627cba2640fdb90f9d237f108268)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.238023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250472bd-4be8-4c19-901a-0208c3528315 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.239636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 38/80] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/users/1aa1627cba2640fdb90f9d237f108268 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:01.244093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250472bd-4be8-4c19-901a-0208c3528315 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.325513 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439aea46-9413-4195-9caf-29786aa6243c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.338514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250472bd-4be8-4c19-901a-0208c3528315 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.359453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038622f-d8f1-43bc-8fec-13935dfad981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.360314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038622f-d8f1-43bc-8fec-13935dfad981 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.360639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038622f-d8f1-43bc-8fec-13935dfad981 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.360921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038622f-d8f1-43bc-8fec-13935dfad981 None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.362114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038622f-d8f1-43bc-8fec-13935dfad981 None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.363570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038622f-d8f1-43bc-8fec-13935dfad981 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.365218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 43/81] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:01.365982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038622f-d8f1-43bc-8fec-13935dfad981 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.374498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9b72-891b-46cd-91ed-fac3c12176f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.375288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9b72-891b-46cd-91ed-fac3c12176f8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.375577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9b72-891b-46cd-91ed-fac3c12176f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.375859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9b72-891b-46cd-91ed-fac3c12176f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.377062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc9b72-891b-46cd-91ed-fac3c12176f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.378477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc9b72-891b-46cd-91ed-fac3c12176f8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.380195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 39/82] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:01.385621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9b72-891b-46cd-91ed-fac3c12176f8 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.428214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.441704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.468313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038622f-d8f1-43bc-8fec-13935dfad981 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.484182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9b72-891b-46cd-91ed-fac3c12176f8 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.502500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe51ca2-884d-43f7-8927-8a4639bb0609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.503147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51ca2-884d-43f7-8927-8a4639bb0609 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.503436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51ca2-884d-43f7-8927-8a4639bb0609 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.503716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51ca2-884d-43f7-8927-8a4639bb0609 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.518769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.519443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.519736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.520023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.520806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:01.521198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.522573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.530485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 40/83] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:01.531355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.570568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe51ca2-884d-43f7-8927-8a4639bb0609 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.572630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe51ca2-884d-43f7-8927-8a4639bb0609 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.628952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.630146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9b827-edb7-4c94-963f-1d178d9e67f9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.647257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.663579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371c5c5-91f7-4c82-ada9-dbf10a8919f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.664227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c5c5-91f7-4c82-ada9-dbf10a8919f9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.664517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c5c5-91f7-4c82-ada9-dbf10a8919f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.664858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c5c5-91f7-4c82-ada9-dbf10a8919f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.669768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 44/84] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:01.670581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe51ca2-884d-43f7-8927-8a4639bb0609 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.733877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371c5c5-91f7-4c82-ada9-dbf10a8919f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.735349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371c5c5-91f7-4c82-ada9-dbf10a8919f9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.768363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe51ca2-884d-43f7-8927-8a4639bb0609 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.801649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba304ddc-4e97-41da-9bfb-dc74c9a9a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.802399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba304ddc-4e97-41da-9bfb-dc74c9a9a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.802695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba304ddc-4e97-41da-9bfb-dc74c9a9a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.802980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba304ddc-4e97-41da-9bfb-dc74c9a9a8f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.830376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.832045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 41/85] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:01.832909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371c5c5-91f7-4c82-ada9-dbf10a8919f9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.850937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.868985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba304ddc-4e97-41da-9bfb-dc74c9a9a8f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.871015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba304ddc-4e97-41da-9bfb-dc74c9a9a8f6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.930605 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371c5c5-91f7-4c82-ada9-dbf10a8919f9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.964038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1460836-59c2-47a2-8792-77b574a9f23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.964751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1460836-59c2-47a2-8792-77b574a9f23f None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.965039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1460836-59c2-47a2-8792-77b574a9f23f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.965315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1460836-59c2-47a2-8792-77b574a9f23f None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.966038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1460836-59c2-47a2-8792-77b574a9f23f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.967476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1460836-59c2-47a2-8792-77b574a9f23f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.969649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 45/86] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:01.970482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba304ddc-4e97-41da-9bfb-dc74c9a9a8f6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.028558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:02.054469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 42/87] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:01 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:02.058416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1460836-59c2-47a2-8792-77b574a9f23f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.071291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba304ddc-4e97-41da-9bfb-dc74c9a9a8f6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.114532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356e533-a2c2-4106-b965-86342d08bbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:02.115474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356e533-a2c2-4106-b965-86342d08bbab None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:02.115759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356e533-a2c2-4106-b965-86342d08bbab None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:02.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356e533-a2c2-4106-b965-86342d08bbab None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:02.117367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356e533-a2c2-4106-b965-86342d08bbab None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:02.119471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356e533-a2c2-4106-b965-86342d08bbab None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:02.125781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 46/88] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:02.125781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356e533-a2c2-4106-b965-86342d08bbab None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.146252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:02.172565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1460836-59c2-47a2-8792-77b574a9f23f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.210131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33d8a1-2a81-437a-bbe3-ca13ab453628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:02.211013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d8a1-2a81-437a-bbe3-ca13ab453628 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:02.211301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d8a1-2a81-437a-bbe3-ca13ab453628 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:02.211583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d8a1-2a81-437a-bbe3-ca13ab453628 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:02.214173 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33d8a1-2a81-437a-bbe3-ca13ab453628 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:02.215625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33d8a1-2a81-437a-bbe3-ca13ab453628 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:02.217742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 43/89] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:02.218128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33d8a1-2a81-437a-bbe3-ca13ab453628 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.238557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356e533-a2c2-4106-b965-86342d08bbab None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.272005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:02.286621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f623e7-9bc0-44b3-8524-d6c3df4b368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:02.286621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f623e7-9bc0-44b3-8524-d6c3df4b368d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:02.286621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f623e7-9bc0-44b3-8524-d6c3df4b368d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:02.286621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f623e7-9bc0-44b3-8524-d6c3df4b368d None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:02.286965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f623e7-9bc0-44b3-8524-d6c3df4b368d None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:02.287965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f623e7-9bc0-44b3-8524-d6c3df4b368d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:02.289528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 47/90] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:02.290288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f623e7-9bc0-44b3-8524-d6c3df4b368d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.337068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33d8a1-2a81-437a-bbe3-ca13ab453628 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.378579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729f121-d5b0-40d3-aacf-91ae701e2e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:02.379731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729f121-d5b0-40d3-aacf-91ae701e2e33 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:02.380021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729f121-d5b0-40d3-aacf-91ae701e2e33 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:02.380308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729f121-d5b0-40d3-aacf-91ae701e2e33 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:02.381921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729f121-d5b0-40d3-aacf-91ae701e2e33 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:02.382972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729f121-d5b0-40d3-aacf-91ae701e2e33 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:02.385394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 44/91] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:02.385503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729f121-d5b0-40d3-aacf-91ae701e2e33 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.400805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:02.411574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f623e7-9bc0-44b3-8524-d6c3df4b368d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.455010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b05c9b-29cf-4178-aa6c-8ea59ab55e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:02.456051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b05c9b-29cf-4178-aa6c-8ea59ab55e11 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:02.456801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b05c9b-29cf-4178-aa6c-8ea59ab55e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:02.457322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b05c9b-29cf-4178-aa6c-8ea59ab55e11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:02.500846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729f121-d5b0-40d3-aacf-91ae701e2e33 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.533284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:02.545152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b05c9b-29cf-4178-aa6c-8ea59ab55e11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:02.545152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b05c9b-29cf-4178-aa6c-8ea59ab55e11 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:02.550876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:02.552197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:02.552725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:02.553561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:02.554921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] RBAC: Adding query filter params (role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=bcf793428063417f9cb53b3d0b4eea85, user.id=714d13e3ae3b43d2a5b81b6f4fc5d980) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:02.555841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:02.557720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:02.570004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 45/92] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/role_assignments?user.id=714d13e3ae3b43d2a5b81b6f4fc5d980&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:02.571313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.670799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:02.706891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a64a56-e918-4838-9a40-4618b27464e9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.748383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 48/93] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:02.750586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b05c9b-29cf-4178-aa6c-8ea59ab55e11 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.784332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:02.785055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:02.785404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:02.785692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:02.786655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=bcf793428063417f9cb53b3d0b4eea85, user.id=1aa1627cba2640fdb90f9d237f108268) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:02.786978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:02.789465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:02.800729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 46/94] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/role_assignments?user.id=1aa1627cba2640fdb90f9d237f108268&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:02.801540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.925004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86d37d-c3f0-4365-852c-894b9f66ff75 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.967363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7277f8-fb51-46d2-bc52-fb7f477db3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:02.968562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7277f8-fb51-46d2-bc52-fb7f477db3ed None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:02.968990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7277f8-fb51-46d2-bc52-fb7f477db3ed None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:02.969433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7277f8-fb51-46d2-bc52-fb7f477db3ed None admin] PATH_INFO: `/v3/projects/111c2793aa2941e8a0189c72bae17e52` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:02.970821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7277f8-fb51-46d2-bc52-fb7f477db3ed None admin] RBAC: Authorizing `identity:get_project(project_id=111c2793aa2941e8a0189c72bae17e52)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:02.972419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7277f8-fb51-46d2-bc52-fb7f477db3ed None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:02.974097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:02.976125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 47/95] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/projects/111c2793aa2941e8a0189c72bae17e52 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:02.982945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7277f8-fb51-46d2-bc52-fb7f477db3ed None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:02.992800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b05c9b-29cf-4178-aa6c-8ea59ab55e11 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:02.999308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:03.093445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a64f6-d6ab-4f92-849e-8177b7feb2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:03.094048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a64f6-d6ab-4f92-849e-8177b7feb2fb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:03.094335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a64f6-d6ab-4f92-849e-8177b7feb2fb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:03.094615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a64f6-d6ab-4f92-849e-8177b7feb2fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:03.095471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a64f6-d6ab-4f92-849e-8177b7feb2fb None admin] RBAC: Adding query filter params (role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=bcf793428063417f9cb53b3d0b4eea85, user.id=714d13e3ae3b43d2a5b81b6f4fc5d980) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:03.095786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a64f6-d6ab-4f92-849e-8177b7feb2fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:03.097313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a64f6-d6ab-4f92-849e-8177b7feb2fb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:03.107820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 49/96] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/role_assignments?user.id=714d13e3ae3b43d2a5b81b6f4fc5d980&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=64517449a51047a3b2ce1079b4912f57 => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:03.195629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7277f8-fb51-46d2-bc52-fb7f477db3ed None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:03.242674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911178-65dc-4986-ba3f-9d1c17bd25b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:03.244655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911178-65dc-4986-ba3f-9d1c17bd25b3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:03.245086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911178-65dc-4986-ba3f-9d1c17bd25b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:03.245544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911178-65dc-4986-ba3f-9d1c17bd25b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:03.246565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07911178-65dc-4986-ba3f-9d1c17bd25b3 None admin] RBAC: Adding query filter params (role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=111c2793aa2941e8a0189c72bae17e52, user.id=714d13e3ae3b43d2a5b81b6f4fc5d980) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:03.247032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07911178-65dc-4986-ba3f-9d1c17bd25b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:03.248713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07911178-65dc-4986-ba3f-9d1c17bd25b3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:03.257589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 48/97] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:02 2022] GET /identity/v3/role_assignments?user.id=714d13e3ae3b43d2a5b81b6f4fc5d980&scope.project.id=111c2793aa2941e8a0189c72bae17e52&role.id=64517449a51047a3b2ce1079b4912f57 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:04.776922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d243c-565d-465d-8497-33a11abeec17 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.777943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d243c-565d-465d-8497-33a11abeec17 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.778438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d243c-565d-465d-8497-33a11abeec17 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.780090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 50/98] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:04.786030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559a28-fe45-441f-9369-39bc71517ee4 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.786534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559a28-fe45-441f-9369-39bc71517ee4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.786634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559a28-fe45-441f-9369-39bc71517ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.809255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-0f559a28-fe45-441f-9369-39bc71517ee4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:04.810450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-0f559a28-fe45-441f-9369-39bc71517ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:04.899047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:04.902434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a52d60-7f95-4e60-93b1-d321ed6b4a44 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.902434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a52d60-7f95-4e60-93b1-d321ed6b4a44 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.902775 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a52d60-7f95-4e60-93b1-d321ed6b4a44 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.904218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 51/99] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:04.905169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 49/100] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:04.912614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5c42-d22d-426d-be54-703253402383 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.912909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5c42-d22d-426d-be54-703253402383 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.913196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5c42-d22d-426d-be54-703253402383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.945524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-8f7e5c42-d22d-426d-be54-703253402383 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:04.946820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8f7e5c42-d22d-426d-be54-703253402383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:04.947665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f559a28-fe45-441f-9369-39bc71517ee4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:05.079559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 52/101] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:05.119121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f559a28-fe45-441f-9369-39bc71517ee4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:05.127377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5c42-d22d-426d-be54-703253402383 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:05.178155 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef53c25-ab93-459d-9b05-58ccad0d7317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:05.179300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef53c25-ab93-459d-9b05-58ccad0d7317 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:05.179780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef53c25-ab93-459d-9b05-58ccad0d7317 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:05.180334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef53c25-ab93-459d-9b05-58ccad0d7317 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:05.205442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef53c25-ab93-459d-9b05-58ccad0d7317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3ef53c25-ab93-459d-9b05-58ccad0d7317 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:05.213738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:05.214643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:05.215431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:05.220608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 50/102] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:05.222896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef53c25-ab93-459d-9b05-58ccad0d7317 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:05.308649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:05.327660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5c42-d22d-426d-be54-703253402383 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:05.356835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:05.421258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54416af-4444-4b02-bd31-64e74b8bd596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:05.422078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54416af-4444-4b02-bd31-64e74b8bd596 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:05.422631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54416af-4444-4b02-bd31-64e74b8bd596 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:05.422911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54416af-4444-4b02-bd31-64e74b8bd596 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:05.424109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54416af-4444-4b02-bd31-64e74b8bd596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:05.425599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54416af-4444-4b02-bd31-64e74b8bd596 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:05.427392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 53/103] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:05.433598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54416af-4444-4b02-bd31-64e74b8bd596 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:05.434763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef53c25-ab93-459d-9b05-58ccad0d7317 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:05.501793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:05.503389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:05.504025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:05.504503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:05.505573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:05.506076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:05.507642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:05.527211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 51/104] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:05.532872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:05.649380 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54416af-4444-4b02-bd31-64e74b8bd596 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:05.718818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c831ff-9943-415a-8078-600e2da9f249 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:05.729188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3ebc-d607-478d-8e93-c0f0c7f2c1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:05.729871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3ebc-d607-478d-8e93-c0f0c7f2c1f3 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:05.730155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3ebc-d607-478d-8e93-c0f0c7f2c1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:05.730537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3ebc-d607-478d-8e93-c0f0c7f2c1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:05.731326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c3ebc-d607-478d-8e93-c0f0c7f2c1f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:05.736552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c3ebc-d607-478d-8e93-c0f0c7f2c1f3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:05.829158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:05.841070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba20cb1-fc77-458f-a78a-833f98e878bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:05.841961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba20cb1-fc77-458f-a78a-833f98e878bf None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:05.842251 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba20cb1-fc77-458f-a78a-833f98e878bf None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:05.842540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba20cb1-fc77-458f-a78a-833f98e878bf None admin] PATH_INFO: `/v3/users/e0c5befcce06499a8ee42cf582c22b58` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:05.842929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 54/105] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:16:05 2022] POST /identity/v3/projects => generated 302 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:16:05.844304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cd9bc-7b38-4cf7-ba62-6dbb5e7195a7 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:05.844620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cd9bc-7b38-4cf7-ba62-6dbb5e7195a7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:05.845134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cd9bc-7b38-4cf7-ba62-6dbb5e7195a7 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:05.846842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 55/106] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:05.864902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c1b50-6ecf-4cb7-82f4-d69a899f6728 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:05.865507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c1b50-6ecf-4cb7-82f4-d69a899f6728 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:05.869461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c1b50-6ecf-4cb7-82f4-d69a899f6728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:05.869583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba20cb1-fc77-458f-a78a-833f98e878bf None admin] RBAC: Authorizing `identity:get_user(user_id=e0c5befcce06499a8ee42cf582c22b58)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:05.869583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba20cb1-fc77-458f-a78a-833f98e878bf None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:05.870968 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 52/107] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:05 2022] GET /identity/v3/users/e0c5befcce06499a8ee42cf582c22b58 => generated 289 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:05.876629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba20cb1-fc77-458f-a78a-833f98e878bf None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:05.901909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a21c1b50-6ecf-4cb7-82f4-d69a899f6728 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:05.903668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a21c1b50-6ecf-4cb7-82f4-d69a899f6728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:05.907270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:06.023605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 56/108] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:06.058792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c1b50-6ecf-4cb7-82f4-d69a899f6728 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:06.089845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba20cb1-fc77-458f-a78a-833f98e878bf None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:06.160271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa30b9f-8a43-4acc-ade3-d07d32bb7dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:06.160984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30b9f-8a43-4acc-ade3-d07d32bb7dc8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.161281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30b9f-8a43-4acc-ade3-d07d32bb7dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.161570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30b9f-8a43-4acc-ade3-d07d32bb7dc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.191739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c1b50-6ecf-4cb7-82f4-d69a899f6728 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:06.220761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:06.235087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9ea55-4637-4b16-b731-40e7cb5da2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:06.236131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ea55-4637-4b16-b731-40e7cb5da2d4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.236616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ea55-4637-4b16-b731-40e7cb5da2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.237111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ea55-4637-4b16-b731-40e7cb5da2d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.245036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa30b9f-8a43-4acc-ade3-d07d32bb7dc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:06.246422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:06.246847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa30b9f-8a43-4acc-ade3-d07d32bb7dc8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:06.250175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9ea55-4637-4b16-b731-40e7cb5da2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-89d9ea55-4637-4b16-b731-40e7cb5da2d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:06.260431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:06.261355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:06.262175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:06.264888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 57/109] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:06.266686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa8b69-04e4-4de3-9cc4-d494c80411e9 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.267263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa8b69-04e4-4de3-9cc4-d494c80411e9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.267838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa8b69-04e4-4de3-9cc4-d494c80411e9 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.269409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 58/110] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:06.274586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa8b69-04e4-4de3-9cc4-d494c80411e9 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:06.367730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 53/111] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:16:06.371831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f746c8-1bac-4e99-908c-a7e86eb686a0 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.371831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f746c8-1bac-4e99-908c-a7e86eb686a0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.371831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f746c8-1bac-4e99-908c-a7e86eb686a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.399787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-99f746c8-1bac-4e99-908c-a7e86eb686a0 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:06.401554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-99f746c8-1bac-4e99-908c-a7e86eb686a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:06.430174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa8b69-04e4-4de3-9cc4-d494c80411e9 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:06.473364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:06.474038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.474327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.474613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.475374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:06.475683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:06.477223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:06.486359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 59/112] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:06.487281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:06.491565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 54/113] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:06.493022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3c63-bc0b-42b9-8bf6-b186cbf4cd0e None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.493314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3c63-bc0b-42b9-8bf6-b186cbf4cd0e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.493595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3c63-bc0b-42b9-8bf6-b186cbf4cd0e None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.495615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 55/114] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:06.496225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa3c63-bc0b-42b9-8bf6-b186cbf4cd0e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:06.516896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:06.562476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:06.608508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae63abf-704c-40e0-82cc-277ce0357d47 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:06.636544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa3c63-bc0b-42b9-8bf6-b186cbf4cd0e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:06.647501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efe2ee-08fd-4fc3-9961-f3f06c0d68fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:06.648505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe2ee-08fd-4fc3-9961-f3f06c0d68fb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.648928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe2ee-08fd-4fc3-9961-f3f06c0d68fb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.649451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe2ee-08fd-4fc3-9961-f3f06c0d68fb None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.650845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efe2ee-08fd-4fc3-9961-f3f06c0d68fb None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:06.652487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efe2ee-08fd-4fc3-9961-f3f06c0d68fb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:06.654502 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 60/115] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:06 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:06.655826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d8eb-51d6-48bb-a416-a76c9dc1a03b None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.656268 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d8eb-51d6-48bb-a416-a76c9dc1a03b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.656656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d8eb-51d6-48bb-a416-a76c9dc1a03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.673680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-8791d8eb-51d6-48bb-a416-a76c9dc1a03b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:06.674867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8791d8eb-51d6-48bb-a416-a76c9dc1a03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:06.675037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b211c-edf6-4b90-9337-1ac1ce3969c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:06.676449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b211c-edf6-4b90-9337-1ac1ce3969c6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.676915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b211c-edf6-4b90-9337-1ac1ce3969c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.677196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b211c-edf6-4b90-9337-1ac1ce3969c6 None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.678599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b211c-edf6-4b90-9337-1ac1ce3969c6 None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:06.680053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b211c-edf6-4b90-9337-1ac1ce3969c6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:06.685053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 56/116] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:06 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:06.685053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b211c-edf6-4b90-9337-1ac1ce3969c6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:06.752557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 61/117] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:06.752557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6d5cb-181d-4df3-b746-d54b7a9e0f6e None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.752557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6d5cb-181d-4df3-b746-d54b7a9e0f6e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.752557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6d5cb-181d-4df3-b746-d54b7a9e0f6e None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.752557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 62/118] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:06.752557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6d5cb-181d-4df3-b746-d54b7a9e0f6e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:06.786962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:06.809562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b211c-edf6-4b90-9337-1ac1ce3969c6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:06.818896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:06.846702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8985d36-a037-4eb3-a768-9223684d902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:06.847535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8985d36-a037-4eb3-a768-9223684d902d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.847820 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8985d36-a037-4eb3-a768-9223684d902d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.848108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8985d36-a037-4eb3-a768-9223684d902d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.859359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8985d36-a037-4eb3-a768-9223684d902d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:06.864652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6d5cb-181d-4df3-b746-d54b7a9e0f6e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-c8985d36-a037-4eb3-a768-9223684d902d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:06.867067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:06.867940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:06.868798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 57/119] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:06.869457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8985d36-a037-4eb3-a768-9223684d902d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:06.903804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf4f49-15fe-42d0-a8da-ce3f3b77afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:06.904816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf4f49-15fe-42d0-a8da-ce3f3b77afeb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.905188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf4f49-15fe-42d0-a8da-ce3f3b77afeb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.905470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf4f49-15fe-42d0-a8da-ce3f3b77afeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.906313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf4f49-15fe-42d0-a8da-ce3f3b77afeb None admin] RBAC: Adding query filter params (role.id=062e902156a0420f882cc116e074cc88, scope.project.id=bcf793428063417f9cb53b3d0b4eea85, user.id=e0c5befcce06499a8ee42cf582c22b58) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:06.906628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf4f49-15fe-42d0-a8da-ce3f3b77afeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:06.908076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf4f49-15fe-42d0-a8da-ce3f3b77afeb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:06.913685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 63/120] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:06 2022] GET /identity/v3/role_assignments?user.id=e0c5befcce06499a8ee42cf582c22b58&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=062e902156a0420f882cc116e074cc88 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:06.914899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a177-723c-4bcc-8611-33be8323b2c2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.915193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a177-723c-4bcc-8611-33be8323b2c2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.915476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a177-723c-4bcc-8611-33be8323b2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.938433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a2b2a177-723c-4bcc-8611-33be8323b2c2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:06.939747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a2b2a177-723c-4bcc-8611-33be8323b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:06.995426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8985d36-a037-4eb3-a768-9223684d902d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.021623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 64/121] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:07.028058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2a177-723c-4bcc-8611-33be8323b2c2 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.035639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770b811-e82c-4183-9e61-5fc586a7fa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.037138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770b811-e82c-4183-9e61-5fc586a7fa36 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.037823 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770b811-e82c-4183-9e61-5fc586a7fa36 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.038608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770b811-e82c-4183-9e61-5fc586a7fa36 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.040951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770b811-e82c-4183-9e61-5fc586a7fa36 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:07.042670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770b811-e82c-4183-9e61-5fc586a7fa36 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:07.044875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 58/122] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:06 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:07.045959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770b811-e82c-4183-9e61-5fc586a7fa36 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.064642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.117838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.164434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2a177-723c-4bcc-8611-33be8323b2c2 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.185830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770b811-e82c-4183-9e61-5fc586a7fa36 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.201487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54443646-6f9f-4f61-a9a0-929ea5addf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.202333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54443646-6f9f-4f61-a9a0-929ea5addf33 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.202643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54443646-6f9f-4f61-a9a0-929ea5addf33 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.202923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54443646-6f9f-4f61-a9a0-929ea5addf33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.213959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54443646-6f9f-4f61-a9a0-929ea5addf33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-54443646-6f9f-4f61-a9a0-929ea5addf33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:07.221583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:07.222443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:07.223218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:07.227440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 65/123] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:07.229444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54443646-6f9f-4f61-a9a0-929ea5addf33 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.233296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.234847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.235147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.235432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.236263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:07.236608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:07.238000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:07.252680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 59/124] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:07.252680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.325876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.371809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54443646-6f9f-4f61-a9a0-929ea5addf33 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.375889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7445e58-ea46-4f01-8069-3f8050f75f59 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.392748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.419756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da1cdb-c4a8-45da-b28c-973fbabdd3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.420730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da1cdb-c4a8-45da-b28c-973fbabdd3be None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.420956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da1cdb-c4a8-45da-b28c-973fbabdd3be None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.421328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da1cdb-c4a8-45da-b28c-973fbabdd3be None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85/users/714d13e3ae3b43d2a5b81b6f4fc5d980/roles/96431666cb824a7087dbd9f0e30c0c80` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.423899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da1cdb-c4a8-45da-b28c-973fbabdd3be None admin] RBAC: Authorizing `identity:create_grant(project_id=bcf793428063417f9cb53b3d0b4eea85, user_id=714d13e3ae3b43d2a5b81b6f4fc5d980, role_id=96431666cb824a7087dbd9f0e30c0c80)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:07.425891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da1cdb-c4a8-45da-b28c-973fbabdd3be None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:07.426769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e6d33-6dd1-4bc1-9fee-6ed506cb497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.427725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e6d33-6dd1-4bc1-9fee-6ed506cb497f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.428145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e6d33-6dd1-4bc1-9fee-6ed506cb497f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.428586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e6d33-6dd1-4bc1-9fee-6ed506cb497f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.441874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e6d33-6dd1-4bc1-9fee-6ed506cb497f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-130e6d33-6dd1-4bc1-9fee-6ed506cb497f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:07.449699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:07.450558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:07.451375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 60/125] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:07.452034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e6d33-6dd1-4bc1-9fee-6ed506cb497f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.496553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 66/126] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:07 2022] PUT /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85/users/714d13e3ae3b43d2a5b81b6f4fc5d980/roles/96431666cb824a7087dbd9f0e30c0c80 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:07.497406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da1cdb-c4a8-45da-b28c-973fbabdd3be None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.629061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da1cdb-c4a8-45da-b28c-973fbabdd3be None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.633502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e6d33-6dd1-4bc1-9fee-6ed506cb497f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.669715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.678925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea72e3-fd45-4a5d-b4cf-c8af326d295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.679225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.679891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea72e3-fd45-4a5d-b4cf-c8af326d295f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.679992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.680191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea72e3-fd45-4a5d-b4cf-c8af326d295f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.680289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.680473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.680602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea72e3-fd45-4a5d-b4cf-c8af326d295f None admin] PATH_INFO: `/v3/users/1aa1627cba2640fdb90f9d237f108268` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.681293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] RBAC: Adding query filter params (name=member) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:07.681608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:07.681763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea72e3-fd45-4a5d-b4cf-c8af326d295f None admin] RBAC: Authorizing `identity:get_user(user_id=1aa1627cba2640fdb90f9d237f108268)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:07.683232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:07.683396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea72e3-fd45-4a5d-b4cf-c8af326d295f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:07.685201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 67/127] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:07 2022] GET /identity/v3/users/1aa1627cba2640fdb90f9d237f108268 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:07.686341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea72e3-fd45-4a5d-b4cf-c8af326d295f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.697651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 61/128] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:07.698828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.740743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.818813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea72e3-fd45-4a5d-b4cf-c8af326d295f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.828808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850e281-3e4e-4404-aca8-6c01b97b172f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.864577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.865635 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.866151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.866249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.866961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:07.867383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:07.869362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8b4f4-2c84-4a77-b527-994f01b9d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.869868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:07.870239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8b4f4-2c84-4a77-b527-994f01b9d771 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.870573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8b4f4-2c84-4a77-b527-994f01b9d771 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.870959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8b4f4-2c84-4a77-b527-994f01b9d771 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.872248 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8b4f4-2c84-4a77-b527-994f01b9d771 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:07.873729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8b4f4-2c84-4a77-b527-994f01b9d771 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:07.875872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 62/129] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:07 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:07.878153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 68/130] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:07.880775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8b4f4-2c84-4a77-b527-994f01b9d771 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.882358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.946717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.989579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8b4f4-2c84-4a77-b527-994f01b9d771 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:08.000329 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a11ef0-3ecb-40e8-a8de-bb7f43abc5aa None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:08.003152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:08.025638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547e870-3d03-49e4-886e-dad095903b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:08.026522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547e870-3d03-49e4-886e-dad095903b6e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.026812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547e870-3d03-49e4-886e-dad095903b6e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.027118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547e870-3d03-49e4-886e-dad095903b6e None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.028848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547e870-3d03-49e4-886e-dad095903b6e None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:08.030070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547e870-3d03-49e4-886e-dad095903b6e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:08.031885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 63/131] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:07 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:08.032488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547e870-3d03-49e4-886e-dad095903b6e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:08.055300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085fa54-b315-4181-a9f0-339763c22628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:08.057131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085fa54-b315-4181-a9f0-339763c22628 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.057620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085fa54-b315-4181-a9f0-339763c22628 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.058135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085fa54-b315-4181-a9f0-339763c22628 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85/users/1aa1627cba2640fdb90f9d237f108268/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.063134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085fa54-b315-4181-a9f0-339763c22628 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcf793428063417f9cb53b3d0b4eea85, user_id=1aa1627cba2640fdb90f9d237f108268, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:08.064969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085fa54-b315-4181-a9f0-339763c22628 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:08.140553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547e870-3d03-49e4-886e-dad095903b6e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:08.155479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 69/132] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:07 2022] PUT /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85/users/1aa1627cba2640fdb90f9d237f108268/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:08.161033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085fa54-b315-4181-a9f0-339763c22628 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:08.170913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:08.180064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82557432-98ba-4e26-8f30-59b731038037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:08.181025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82557432-98ba-4e26-8f30-59b731038037 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.181994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82557432-98ba-4e26-8f30-59b731038037 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.182294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82557432-98ba-4e26-8f30-59b731038037 None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.183708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82557432-98ba-4e26-8f30-59b731038037 None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:08.185470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82557432-98ba-4e26-8f30-59b731038037 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:08.187394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 64/133] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:08 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:08.193640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82557432-98ba-4e26-8f30-59b731038037 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:08.312711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085fa54-b315-4181-a9f0-339763c22628 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:08.349285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21591bbd-b6fe-4890-9a85-43f835a5c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:08.350567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21591bbd-b6fe-4890-9a85-43f835a5c374 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.351051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21591bbd-b6fe-4890-9a85-43f835a5c374 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.351523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21591bbd-b6fe-4890-9a85-43f835a5c374 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.353048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21591bbd-b6fe-4890-9a85-43f835a5c374 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:08.355275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21591bbd-b6fe-4890-9a85-43f835a5c374 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:08.357251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 70/134] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:08 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:08.363023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21591bbd-b6fe-4890-9a85-43f835a5c374 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:08.377973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82557432-98ba-4e26-8f30-59b731038037 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:08.396179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:08.460578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba81c9-0ad3-428c-a77c-e6fe0a0194cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:08.461535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba81c9-0ad3-428c-a77c-e6fe0a0194cc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.461833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba81c9-0ad3-428c-a77c-e6fe0a0194cc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.462158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba81c9-0ad3-428c-a77c-e6fe0a0194cc None admin] PATH_INFO: `/v3/projects/111c2793aa2941e8a0189c72bae17e52` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.465715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba81c9-0ad3-428c-a77c-e6fe0a0194cc None admin] RBAC: Authorizing `identity:get_project(project_id=111c2793aa2941e8a0189c72bae17e52)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:08.467079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba81c9-0ad3-428c-a77c-e6fe0a0194cc None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:08.469384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 65/135] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:08 2022] GET /identity/v3/projects/111c2793aa2941e8a0189c72bae17e52 => generated 312 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:08.472249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21591bbd-b6fe-4890-9a85-43f835a5c374 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:08.474623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba81c9-0ad3-428c-a77c-e6fe0a0194cc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:08.515795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a04d7-6560-4268-b6ab-432424bacce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:08.517108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a04d7-6560-4268-b6ab-432424bacce8 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.517589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a04d7-6560-4268-b6ab-432424bacce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.518329 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a04d7-6560-4268-b6ab-432424bacce8 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85/users/714d13e3ae3b43d2a5b81b6f4fc5d980/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.522345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a04d7-6560-4268-b6ab-432424bacce8 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcf793428063417f9cb53b3d0b4eea85, user_id=714d13e3ae3b43d2a5b81b6f4fc5d980, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:08.524637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a04d7-6560-4268-b6ab-432424bacce8 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:08.566611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:08.596867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 71/136] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:08 2022] PUT /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85/users/714d13e3ae3b43d2a5b81b6f4fc5d980/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:16:08.620906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba81c9-0ad3-428c-a77c-e6fe0a0194cc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:08.659714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6de27-429c-49f7-8f6c-912b292e463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:08.660632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6de27-429c-49f7-8f6c-912b292e463f None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.660923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6de27-429c-49f7-8f6c-912b292e463f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.661300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6de27-429c-49f7-8f6c-912b292e463f None admin] PATH_INFO: `/v3/projects/111c2793aa2941e8a0189c72bae17e52/users/714d13e3ae3b43d2a5b81b6f4fc5d980/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.663469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6de27-429c-49f7-8f6c-912b292e463f None admin] RBAC: Authorizing `identity:create_grant(project_id=111c2793aa2941e8a0189c72bae17e52, user_id=714d13e3ae3b43d2a5b81b6f4fc5d980, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:08.665165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6de27-429c-49f7-8f6c-912b292e463f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:08.744370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 66/137] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:08 2022] PUT /identity/v3/projects/111c2793aa2941e8a0189c72bae17e52/users/714d13e3ae3b43d2a5b81b6f4fc5d980/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:16:08.856375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce29c-c2cd-46db-a96a-619e3782adcf None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.857099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce29c-c2cd-46db-a96a-619e3782adcf None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.857585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce29c-c2cd-46db-a96a-619e3782adcf None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.859245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 72/138] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:08.871552 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42302a-3703-492d-b818-7585fabe34b2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:08.872044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42302a-3703-492d-b818-7585fabe34b2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:08.872136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42302a-3703-492d-b818-7585fabe34b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:08.895745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-1d42302a-3703-492d-b818-7585fabe34b2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:08.896989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1d42302a-3703-492d-b818-7585fabe34b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:08.983322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 67/139] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:09.029361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ce29c-c2cd-46db-a96a-619e3782adcf None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:09.157676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:09.167569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ce29c-c2cd-46db-a96a-619e3782adcf None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:09.205035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40225f-2fa9-4209-af47-8bf2828b4f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:09.205848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40225f-2fa9-4209-af47-8bf2828b4f3b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:09.206170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40225f-2fa9-4209-af47-8bf2828b4f3b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:09.206411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40225f-2fa9-4209-af47-8bf2828b4f3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:09.207606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40225f-2fa9-4209-af47-8bf2828b4f3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:09.209156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40225f-2fa9-4209-af47-8bf2828b4f3b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:09.210899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 73/140] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:09.217227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42302a-3703-492d-b818-7585fabe34b2 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:09.309278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:09.359591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42302a-3703-492d-b818-7585fabe34b2 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:09.410364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e67b5-1385-44ff-93b5-193957566053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:09.411417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e67b5-1385-44ff-93b5-193957566053 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:09.411924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e67b5-1385-44ff-93b5-193957566053 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:09.412394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e67b5-1385-44ff-93b5-193957566053 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:09.413420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e67b5-1385-44ff-93b5-193957566053 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:09.415148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e67b5-1385-44ff-93b5-193957566053 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:09.535929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 68/141] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:09 2022] POST /identity/v3/users => generated 283 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:09.758125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc30002-27bb-4b1c-9747-311b01f3b0bf None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:09.758758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc30002-27bb-4b1c-9747-311b01f3b0bf None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:09.759266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc30002-27bb-4b1c-9747-311b01f3b0bf None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:09.760894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 74/142] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:09.768341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f870e-d11e-451f-a0f2-b2c16a25498f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:09.768859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f870e-d11e-451f-a0f2-b2c16a25498f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:09.769150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f870e-d11e-451f-a0f2-b2c16a25498f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:09.793599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-2b5f870e-d11e-451f-a0f2-b2c16a25498f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:09.794799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2b5f870e-d11e-451f-a0f2-b2c16a25498f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:09.887480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 69/143] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:09.917299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc30002-27bb-4b1c-9747-311b01f3b0bf None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:10.041776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc30002-27bb-4b1c-9747-311b01f3b0bf None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:10.087661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388b24-a434-4e5b-8252-9f09e832e965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:10.088463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388b24-a434-4e5b-8252-9f09e832e965 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:10.088776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388b24-a434-4e5b-8252-9f09e832e965 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:10.089049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388b24-a434-4e5b-8252-9f09e832e965 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:10.106334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388b24-a434-4e5b-8252-9f09e832e965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-56388b24-a434-4e5b-8252-9f09e832e965 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:10.113986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:10.114864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:10.115640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 75/144] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:10.121746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f870e-d11e-451f-a0f2-b2c16a25498f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:10.199013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:10.270158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f870e-d11e-451f-a0f2-b2c16a25498f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:10.309592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:10.310617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:10.311131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:10.311610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:10.312615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:10.313129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:10.314723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:10.328063 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 70/145] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:10.333554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388b24-a434-4e5b-8252-9f09e832e965 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:10.384903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:10.489259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388b24-a434-4e5b-8252-9f09e832e965 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:10.535219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797f141-69df-4c4e-b633-aeffda71078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:10.536107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797f141-69df-4c4e-b633-aeffda71078c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:10.536392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797f141-69df-4c4e-b633-aeffda71078c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:10.536705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797f141-69df-4c4e-b633-aeffda71078c None admin] PATH_INFO: `/v3/users/e0c5befcce06499a8ee42cf582c22b58` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:10.537942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797f141-69df-4c4e-b633-aeffda71078c None admin] RBAC: Authorizing `identity:get_user(user_id=e0c5befcce06499a8ee42cf582c22b58)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:10.539391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797f141-69df-4c4e-b633-aeffda71078c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:10.541622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 76/146] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:10 2022] GET /identity/v3/users/e0c5befcce06499a8ee42cf582c22b58 => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:10.547234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:10.707686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:10.711128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e272a7-c862-458a-b27a-9796098b9776 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:10.755544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a061599-a030-4f04-8a9b-c5d35f12efca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:10.756832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a061599-a030-4f04-8a9b-c5d35f12efca None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:10.757328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a061599-a030-4f04-8a9b-c5d35f12efca None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:10.757812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a061599-a030-4f04-8a9b-c5d35f12efca None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:10.759316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a061599-a030-4f04-8a9b-c5d35f12efca None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:10.761510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a061599-a030-4f04-8a9b-c5d35f12efca None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:10.763772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 71/147] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:10 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:10.773588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797f141-69df-4c4e-b633-aeffda71078c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:10.783694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27ea79-f9ee-45a4-bd8b-a17d2187ee33 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:10.784288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27ea79-f9ee-45a4-bd8b-a17d2187ee33 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:10.784865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27ea79-f9ee-45a4-bd8b-a17d2187ee33 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:10.786467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 72/148] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:10.792898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5878237-4bd9-4d41-88c7-5fee91312f11 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:10.793558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5878237-4bd9-4d41-88c7-5fee91312f11 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:10.794112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5878237-4bd9-4d41-88c7-5fee91312f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:10.826593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f5878237-4bd9-4d41-88c7-5fee91312f11 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:10.829638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f5878237-4bd9-4d41-88c7-5fee91312f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:10.946693 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 73/149] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:10.992906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:10.998780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5878237-4bd9-4d41-88c7-5fee91312f11 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:11.009882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797f141-69df-4c4e-b633-aeffda71078c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:11.086677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4bcd3-0d67-4f27-b18e-efa3eb385ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:11.087626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4bcd3-0d67-4f27-b18e-efa3eb385ad7 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.087917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4bcd3-0d67-4f27-b18e-efa3eb385ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.088198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4bcd3-0d67-4f27-b18e-efa3eb385ad7 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85/users/e0c5befcce06499a8ee42cf582c22b58/roles/062e902156a0420f882cc116e074cc88` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.093899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4bcd3-0d67-4f27-b18e-efa3eb385ad7 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcf793428063417f9cb53b3d0b4eea85, user_id=e0c5befcce06499a8ee42cf582c22b58, role_id=062e902156a0420f882cc116e074cc88)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:11.096603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4bcd3-0d67-4f27-b18e-efa3eb385ad7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:11.155902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:11.160387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5878237-4bd9-4d41-88c7-5fee91312f11 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:11.186622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 77/150] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:10 2022] PUT /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85/users/e0c5befcce06499a8ee42cf582c22b58/roles/062e902156a0420f882cc116e074cc88 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:11.211480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227f0dd-70cd-41c7-a169-35f508d9c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:11.212447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227f0dd-70cd-41c7-a169-35f508d9c305 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.213279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227f0dd-70cd-41c7-a169-35f508d9c305 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.213580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227f0dd-70cd-41c7-a169-35f508d9c305 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.233642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227f0dd-70cd-41c7-a169-35f508d9c305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-7227f0dd-70cd-41c7-a169-35f508d9c305 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:11.245233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:11.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:11.246965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 74/151] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:11.254571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4bcd3-0d67-4f27-b18e-efa3eb385ad7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:11.378972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4bcd3-0d67-4f27-b18e-efa3eb385ad7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:11.418136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:11.419472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.419768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.420051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.420872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:11.421190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:11.422878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:11.433210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 78/152] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:11.438958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227f0dd-70cd-41c7-a169-35f508d9c305 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:11.568922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227f0dd-70cd-41c7-a169-35f508d9c305 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:11.621817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703d557-0577-47e0-b473-d70272ffc21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:11.622764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703d557-0577-47e0-b473-d70272ffc21f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.623056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703d557-0577-47e0-b473-d70272ffc21f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.623337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703d557-0577-47e0-b473-d70272ffc21f None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.626585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703d557-0577-47e0-b473-d70272ffc21f None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:11.628691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703d557-0577-47e0-b473-d70272ffc21f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:11.630561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 75/153] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:11 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:11.637091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:11.652545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:11.746490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701c82e-f290-4de8-bc8e-71b6396ed47f None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.746796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701c82e-f290-4de8-bc8e-71b6396ed47f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.747062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701c82e-f290-4de8-bc8e-71b6396ed47f None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.748423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 76/154] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:11.757820 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560d906-e979-40c2-8e25-f4cfecfa94e7 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.758108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560d906-e979-40c2-8e25-f4cfecfa94e7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.758391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560d906-e979-40c2-8e25-f4cfecfa94e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.760199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8450cd2-dc10-4ff4-8ea7-bd5cb579cb67 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:11.789224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6560d906-e979-40c2-8e25-f4cfecfa94e7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:11.790421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6560d906-e979-40c2-8e25-f4cfecfa94e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:11.816993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab44297-46a2-48fc-9061-a7f46797935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:11.817677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab44297-46a2-48fc-9061-a7f46797935a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.817935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab44297-46a2-48fc-9061-a7f46797935a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.818216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab44297-46a2-48fc-9061-a7f46797935a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.844935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:11.924478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab44297-46a2-48fc-9061-a7f46797935a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:11.929710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab44297-46a2-48fc-9061-a7f46797935a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:11.946119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:11.982855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 77/155] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 00:16:12.045055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560d906-e979-40c2-8e25-f4cfecfa94e7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:12.128793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 79/156] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:16:12.134105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab44297-46a2-48fc-9061-a7f46797935a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:12.318743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560d906-e979-40c2-8e25-f4cfecfa94e7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:12.319269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab44297-46a2-48fc-9061-a7f46797935a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:12.335106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:12.374903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a975724-4d2f-4539-b3c8-f480f8016a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:12.376143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975724-4d2f-4539-b3c8-f480f8016a93 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.376692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975724-4d2f-4539-b3c8-f480f8016a93 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.377175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975724-4d2f-4539-b3c8-f480f8016a93 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.385561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a975724-4d2f-4539-b3c8-f480f8016a93 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:12.387065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a975724-4d2f-4539-b3c8-f480f8016a93 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:12.389075 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 80/157] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:12 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:12.395127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a975724-4d2f-4539-b3c8-f480f8016a93 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:12.417285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26992ebc-b2cb-41fe-9d81-18a333cc086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:12.418115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26992ebc-b2cb-41fe-9d81-18a333cc086c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.418404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26992ebc-b2cb-41fe-9d81-18a333cc086c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.418688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26992ebc-b2cb-41fe-9d81-18a333cc086c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.445445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26992ebc-b2cb-41fe-9d81-18a333cc086c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-26992ebc-b2cb-41fe-9d81-18a333cc086c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:12.453323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:12.454284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:12.455088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 78/158] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:12.463555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26992ebc-b2cb-41fe-9d81-18a333cc086c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:12.514840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:12.622070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a975724-4d2f-4539-b3c8-f480f8016a93 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:12.681783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf5bd5-7748-457f-9547-462fc61e5282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:12.682464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf5bd5-7748-457f-9547-462fc61e5282 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.682749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf5bd5-7748-457f-9547-462fc61e5282 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.683026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf5bd5-7748-457f-9547-462fc61e5282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.683949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf5bd5-7748-457f-9547-462fc61e5282 None admin] RBAC: Adding query filter params (role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=bcf793428063417f9cb53b3d0b4eea85, user.id=714d13e3ae3b43d2a5b81b6f4fc5d980) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:12.684258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf5bd5-7748-457f-9547-462fc61e5282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:12.685752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf5bd5-7748-457f-9547-462fc61e5282 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:12.692062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 81/159] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:12 2022] GET /identity/v3/role_assignments?user.id=714d13e3ae3b43d2a5b81b6f4fc5d980&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:12.697860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef5b4-50a7-4518-b30a-5f04d34b6c79 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.698115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef5b4-50a7-4518-b30a-5f04d34b6c79 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.698341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef5b4-50a7-4518-b30a-5f04d34b6c79 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.700060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 82/160] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:12.706035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ed71b-3c74-425c-9ef8-b925826a9c50 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.706035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ed71b-3c74-425c-9ef8-b925826a9c50 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.706475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ed71b-3c74-425c-9ef8-b925826a9c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.737965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26992ebc-b2cb-41fe-9d81-18a333cc086c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:12.738702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-461ed71b-3c74-425c-9ef8-b925826a9c50 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:12.739915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-461ed71b-3c74-425c-9ef8-b925826a9c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:12.809175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:12.846601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:12.847204 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.847487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.847762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.848547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:12.848862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:12.850255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:12.853539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 83/161] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:12.854619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283725f-c747-4e85-b630-a801868e187f None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.864995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283725f-c747-4e85-b630-a801868e187f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.864995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283725f-c747-4e85-b630-a801868e187f None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.864995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 84/162] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:12.864995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca6bca-eefc-4425-b26e-d31b62519cc9 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.864995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca6bca-eefc-4425-b26e-d31b62519cc9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.864995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca6bca-eefc-4425-b26e-d31b62519cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.882152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 79/163] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:12.890669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:12.909811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-cfca6bca-eefc-4425-b26e-d31b62519cc9 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:12.911084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-cfca6bca-eefc-4425-b26e-d31b62519cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:13.050022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 85/164] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:13.050916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca6bca-eefc-4425-b26e-d31b62519cc9 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:13.080635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:13.112922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a858a4-3782-44e1-84ac-25eaea8dafe0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:13.151175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8e5bf-b169-4644-8218-1f3328cd7f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:13.152827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8e5bf-b169-4644-8218-1f3328cd7f15 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:13.153273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8e5bf-b169-4644-8218-1f3328cd7f15 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:13.153705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8e5bf-b169-4644-8218-1f3328cd7f15 None admin] PATH_INFO: `/v3/users/1aa1627cba2640fdb90f9d237f108268` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:13.155212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8e5bf-b169-4644-8218-1f3328cd7f15 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa1627cba2640fdb90f9d237f108268)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:13.156817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8e5bf-b169-4644-8218-1f3328cd7f15 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:13.158779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 80/165] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:12 2022] GET /identity/v3/users/1aa1627cba2640fdb90f9d237f108268 => generated 247 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:13.159794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8e5bf-b169-4644-8218-1f3328cd7f15 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:13.240201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca6bca-eefc-4425-b26e-d31b62519cc9 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:13.241969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:13.305271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8e5bf-b169-4644-8218-1f3328cd7f15 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:13.348256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d6f5-7e68-4ae1-b2d6-fee12efdb7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:13.349093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d6f5-7e68-4ae1-b2d6-fee12efdb7ea None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:13.349377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d6f5-7e68-4ae1-b2d6-fee12efdb7ea None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:13.349651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d6f5-7e68-4ae1-b2d6-fee12efdb7ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:13.355759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681cabc-1fa9-48a0-a535-76f00461ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:13.356574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681cabc-1fa9-48a0-a535-76f00461ce3e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:13.356865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681cabc-1fa9-48a0-a535-76f00461ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:13.357142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681cabc-1fa9-48a0-a535-76f00461ce3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:13.373574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681cabc-1fa9-48a0-a535-76f00461ce3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-b681cabc-1fa9-48a0-a535-76f00461ce3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:13.385592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:13.386595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:13.387635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 81/166] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:13.388342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681cabc-1fa9-48a0-a535-76f00461ce3e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:13.393266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6d6f5-7e68-4ae1-b2d6-fee12efdb7ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-c5a6d6f5-7e68-4ae1-b2d6-fee12efdb7ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:13.401801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:13.402828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:13.403664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 86/167] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:13.404335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d6f5-7e68-4ae1-b2d6-fee12efdb7ea None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:13.411203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:13.568230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681cabc-1fa9-48a0-a535-76f00461ce3e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:13.658421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee245cb7-3189-47d1-95ba-ccc46c3e3c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:13.658421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee245cb7-3189-47d1-95ba-ccc46c3e3c01 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:13.658421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee245cb7-3189-47d1-95ba-ccc46c3e3c01 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:13.658421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee245cb7-3189-47d1-95ba-ccc46c3e3c01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:13.658901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d6f5-7e68-4ae1-b2d6-fee12efdb7ea None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:13.748411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:13.749117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:13.749430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:13.749678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:13.750450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:13.750758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:13.752334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:13.770872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 87/168] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:13.771520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:13.809402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee245cb7-3189-47d1-95ba-ccc46c3e3c01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:13.811267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee245cb7-3189-47d1-95ba-ccc46c3e3c01 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:13.823709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:13.854145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.009115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbf05-26e5-45a1-bdf9-214d443099b9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.027647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 82/169] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:14.028546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee245cb7-3189-47d1-95ba-ccc46c3e3c01 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.079032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.080072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.080507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.080963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.081863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:14.083351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.087073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.104584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 88/170] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:14.113570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.174786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee245cb7-3189-47d1-95ba-ccc46c3e3c01 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.184052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.217213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec2d45-0d48-42ec-ba38-8adab9549509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.218349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec2d45-0d48-42ec-ba38-8adab9549509 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.218817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec2d45-0d48-42ec-ba38-8adab9549509 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.219285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec2d45-0d48-42ec-ba38-8adab9549509 None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.222851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec2d45-0d48-42ec-ba38-8adab9549509 None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.224317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec2d45-0d48-42ec-ba38-8adab9549509 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.231432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 83/171] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:14.232269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec2d45-0d48-42ec-ba38-8adab9549509 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.245673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.257974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1dac5-516b-4d28-aa2b-9b097a21e186 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.294516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf02c85-2bac-4659-a6d1-330aba765afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.295303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf02c85-2bac-4659-a6d1-330aba765afe None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.295835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf02c85-2bac-4659-a6d1-330aba765afe None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.295835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf02c85-2bac-4659-a6d1-330aba765afe None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.297393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf02c85-2bac-4659-a6d1-330aba765afe None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.299314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf02c85-2bac-4659-a6d1-330aba765afe None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.301499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 89/172] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:14.302416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf02c85-2bac-4659-a6d1-330aba765afe None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.406443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf02c85-2bac-4659-a6d1-330aba765afe None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.429882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec2d45-0d48-42ec-ba38-8adab9549509 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.445437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afad30-5e08-4d8e-afce-8e66d3a979af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.446104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afad30-5e08-4d8e-afce-8e66d3a979af None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.446392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afad30-5e08-4d8e-afce-8e66d3a979af None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.446724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afad30-5e08-4d8e-afce-8e66d3a979af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.476987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.493314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec06aaf-581f-4850-ae31-850daa55ed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.494773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06aaf-581f-4850-ae31-850daa55ed65 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.495056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06aaf-581f-4850-ae31-850daa55ed65 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.495333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06aaf-581f-4850-ae31-850daa55ed65 None admin] PATH_INFO: `/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.496640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec06aaf-581f-4850-ae31-850daa55ed65 None admin] RBAC: Authorizing `identity:get_user(user_id=714d13e3ae3b43d2a5b81b6f4fc5d980)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.498604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec06aaf-581f-4850-ae31-850daa55ed65 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.500258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 84/173] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/users/714d13e3ae3b43d2a5b81b6f4fc5d980 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:14.501051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec06aaf-581f-4850-ae31-850daa55ed65 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.528061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afad30-5e08-4d8e-afce-8e66d3a979af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.529585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afad30-5e08-4d8e-afce-8e66d3a979af None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.580145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.637853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 90/174] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:14.643145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afad30-5e08-4d8e-afce-8e66d3a979af None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.688434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec06aaf-581f-4850-ae31-850daa55ed65 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.701814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.730838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e492-c712-4203-9f74-97ac06b10cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.732037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e492-c712-4203-9f74-97ac06b10cd6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.732956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e492-c712-4203-9f74-97ac06b10cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.733782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e492-c712-4203-9f74-97ac06b10cd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.735092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66e492-c712-4203-9f74-97ac06b10cd6 None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=bcf793428063417f9cb53b3d0b4eea85, user.id=1aa1627cba2640fdb90f9d237f108268) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:14.735903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66e492-c712-4203-9f74-97ac06b10cd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.737589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66e492-c712-4203-9f74-97ac06b10cd6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.747394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 85/175] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/role_assignments?user.id=1aa1627cba2640fdb90f9d237f108268&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:14.749610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbeaef-761e-4cdf-a5ec-6873e70756d8 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.750403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbeaef-761e-4cdf-a5ec-6873e70756d8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.750924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbeaef-761e-4cdf-a5ec-6873e70756d8 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.751972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afad30-5e08-4d8e-afce-8e66d3a979af None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.753071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 86/176] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:14.754448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbeaef-761e-4cdf-a5ec-6873e70756d8 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.792696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0367d-984e-41f1-acb5-1f07b01ad538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.793362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0367d-984e-41f1-acb5-1f07b01ad538 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.793652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0367d-984e-41f1-acb5-1f07b01ad538 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.793932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0367d-984e-41f1-acb5-1f07b01ad538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.901422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0367d-984e-41f1-acb5-1f07b01ad538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.903466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0367d-984e-41f1-acb5-1f07b01ad538 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.908483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbeaef-761e-4cdf-a5ec-6873e70756d8 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.924736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.990247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de0ac0-f2ec-4061-b20a-4da166fa4973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.991614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de0ac0-f2ec-4061-b20a-4da166fa4973 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.992106 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de0ac0-f2ec-4061-b20a-4da166fa4973 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.992683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de0ac0-f2ec-4061-b20a-4da166fa4973 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.994694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de0ac0-f2ec-4061-b20a-4da166fa4973 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.996425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de0ac0-f2ec-4061-b20a-4da166fa4973 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.998478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 87/177] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:15.000026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be87a-1413-4ca8-9cdf-88b0e1ce78a9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.000552 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be87a-1413-4ca8-9cdf-88b0e1ce78a9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.001055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be87a-1413-4ca8-9cdf-88b0e1ce78a9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.002569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 88/178] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:15.003887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a90bd-7644-42ab-9821-f33049169cf4 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.004376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a90bd-7644-42ab-9821-f33049169cf4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.004922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a90bd-7644-42ab-9821-f33049169cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.022462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.025282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-403a90bd-7644-42ab-9821-f33049169cf4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:15.026694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-403a90bd-7644-42ab-9821-f33049169cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:15.075035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 91/179] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:15.076812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce70b9-65e4-4f83-8a84-c0c2d7a288ba None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.077179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce70b9-65e4-4f83-8a84-c0c2d7a288ba None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.077534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce70b9-65e4-4f83-8a84-c0c2d7a288ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.095742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f1ce70b9-65e4-4f83-8a84-c0c2d7a288ba None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:15.096980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f1ce70b9-65e4-4f83-8a84-c0c2d7a288ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:15.106605 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 89/180] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:15.116702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a90bd-7644-42ab-9821-f33049169cf4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.167726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 92/181] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:15.168991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1510b9-d870-4269-bd7d-e00690a17897 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.169285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1510b9-d870-4269-bd7d-e00690a17897 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.169568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1510b9-d870-4269-bd7d-e00690a17897 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.171001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 93/182] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:15.171926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca769ad-f2d3-433d-9d83-ed624954a106 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.172220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca769ad-f2d3-433d-9d83-ed624954a106 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.172787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca769ad-f2d3-433d-9d83-ed624954a106 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.178026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 94/183] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:15.178026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca769ad-f2d3-433d-9d83-ed624954a106 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.215843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a90bd-7644-42ab-9821-f33049169cf4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.218131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.250465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.251119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.251404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.251680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.252549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] RBAC: Adding query filter params (role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=bcf793428063417f9cb53b3d0b4eea85, user.id=714d13e3ae3b43d2a5b81b6f4fc5d980) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:15.252858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.254297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.260545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 90/184] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/role_assignments?user.id=714d13e3ae3b43d2a5b81b6f4fc5d980&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=64517449a51047a3b2ce1079b4912f57 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:15.261322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.277705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca769ad-f2d3-433d-9d83-ed624954a106 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.296957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.321335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339d36e-125f-4a64-a40b-35caddbc805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.322922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339d36e-125f-4a64-a40b-35caddbc805f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.323226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339d36e-125f-4a64-a40b-35caddbc805f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.323600 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339d36e-125f-4a64-a40b-35caddbc805f None admin] PATH_INFO: `/v3/projects/111c2793aa2941e8a0189c72bae17e52` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.325811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339d36e-125f-4a64-a40b-35caddbc805f None admin] RBAC: Authorizing `identity:get_project(project_id=111c2793aa2941e8a0189c72bae17e52)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.327284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339d36e-125f-4a64-a40b-35caddbc805f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.329086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 95/185] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/projects/111c2793aa2941e8a0189c72bae17e52 => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:15.337358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d57dd9-def9-4f4f-907e-c4d5721f3cce None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.337358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d57dd9-def9-4f4f-907e-c4d5721f3cce None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.337358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d57dd9-def9-4f4f-907e-c4d5721f3cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.348350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e1d57dd9-def9-4f4f-907e-c4d5721f3cce None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:15.349601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e1d57dd9-def9-4f4f-907e-c4d5721f3cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:15.391352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a0af1-6334-4944-b871-21aa9de39684 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.428297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 96/186] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:15.429953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a8e7c-3f91-448d-84d9-2197d6d2131e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.430246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a8e7c-3f91-448d-84d9-2197d6d2131e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.430529 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a8e7c-3f91-448d-84d9-2197d6d2131e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.433304 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca7c70-f3b0-47ae-bc98-c26782616029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.434153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7c70-f3b0-47ae-bc98-c26782616029 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.434392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7c70-f3b0-47ae-bc98-c26782616029 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.434671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7c70-f3b0-47ae-bc98-c26782616029 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.446544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca7c70-f3b0-47ae-bc98-c26782616029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:15.452807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7b2a8e7c-3f91-448d-84d9-2197d6d2131e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:15.454170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7b2a8e7c-3f91-448d-84d9-2197d6d2131e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-caca7c70-f3b0-47ae-bc98-c26782616029 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:15.455112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:15.455996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:15.456854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 91/187] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:15.457545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca7c70-f3b0-47ae-bc98-c26782616029 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.494776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.533465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 97/188] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:15.537407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a8e7c-3f91-448d-84d9-2197d6d2131e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.552182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.573251 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca7c70-f3b0-47ae-bc98-c26782616029 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.608329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278b876-1a44-4143-aa00-d8bf60c49d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.609150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278b876-1a44-4143-aa00-d8bf60c49d73 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.609433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278b876-1a44-4143-aa00-d8bf60c49d73 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.609707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278b876-1a44-4143-aa00-d8bf60c49d73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.620452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278b876-1a44-4143-aa00-d8bf60c49d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-1278b876-1a44-4143-aa00-d8bf60c49d73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:15.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:15.628932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:15.629972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 92/189] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:15.633355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278b876-1a44-4143-aa00-d8bf60c49d73 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.640954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a8e7c-3f91-448d-84d9-2197d6d2131e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.676337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.677038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.677320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.677596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.678499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] RBAC: Adding query filter params (role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=111c2793aa2941e8a0189c72bae17e52, user.id=714d13e3ae3b43d2a5b81b6f4fc5d980) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:15.678759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.682431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.691144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 98/190] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/role_assignments?user.id=714d13e3ae3b43d2a5b81b6f4fc5d980&scope.project.id=111c2793aa2941e8a0189c72bae17e52&role.id=64517449a51047a3b2ce1079b4912f57 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:15.691874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.735510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.748003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278b876-1a44-4143-aa00-d8bf60c49d73 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.784934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796132-8538-4de8-b6aa-954bb3d4c029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.785750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796132-8538-4de8-b6aa-954bb3d4c029 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.786038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796132-8538-4de8-b6aa-954bb3d4c029 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.786317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796132-8538-4de8-b6aa-954bb3d4c029 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.787541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796132-8538-4de8-b6aa-954bb3d4c029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.789035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796132-8538-4de8-b6aa-954bb3d4c029 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.791273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 93/191] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:15.792061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796132-8538-4de8-b6aa-954bb3d4c029 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.803828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.828583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724834b0-cd88-4e37-b5d4-74bc33ee4ecb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.867649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.868329 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.868774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.869031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.869998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:15.870110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.871485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.879749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 99/192] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:15.880675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.916025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796132-8538-4de8-b6aa-954bb3d4c029 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.952080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251bc41-cd8e-48c4-9f98-9f266c4e322a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.952862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251bc41-cd8e-48c4-9f98-9f266c4e322a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.953142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251bc41-cd8e-48c4-9f98-9f266c4e322a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.953414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251bc41-cd8e-48c4-9f98-9f266c4e322a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.963856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251bc41-cd8e-48c4-9f98-9f266c4e322a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-b251bc41-cd8e-48c4-9f98-9f266c4e322a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:15.971518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:15.972510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:15.973365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 94/193] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:15.974373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251bc41-cd8e-48c4-9f98-9f266c4e322a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.989420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f9473-46a4-48d9-809d-dba4a89e1737 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.995828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:16.027313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.027980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.028263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.028567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.029316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:16.029625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.031155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:16.036811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:16.041387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 100/194] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:16.045457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:16.084147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251bc41-cd8e-48c4-9f98-9f266c4e322a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:16.117040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c33aa-db9e-4c29-b998-7487fed65a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.117712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c33aa-db9e-4c29-b998-7487fed65a86 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.117992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c33aa-db9e-4c29-b998-7487fed65a86 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.118277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c33aa-db9e-4c29-b998-7487fed65a86 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.119021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c33aa-db9e-4c29-b998-7487fed65a86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.120486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c33aa-db9e-4c29-b998-7487fed65a86 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:16.145959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c08e2-88bb-42fa-a937-479ddeb93b0d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:16.179341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df9cae-0979-439b-a4ca-e654cfb9f056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.180201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df9cae-0979-439b-a4ca-e654cfb9f056 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.180480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df9cae-0979-439b-a4ca-e654cfb9f056 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.180780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df9cae-0979-439b-a4ca-e654cfb9f056 None admin] PATH_INFO: `/v3/users/7f3bbab298794248859fc6f8b45e2e96` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.191420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 95/195] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:15 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:16.192543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c33aa-db9e-4c29-b998-7487fed65a86 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:16.199815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df9cae-0979-439b-a4ca-e654cfb9f056 None admin] RBAC: Authorizing `identity:get_user(user_id=7f3bbab298794248859fc6f8b45e2e96)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.201382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df9cae-0979-439b-a4ca-e654cfb9f056 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:16.203506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 101/196] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:16 2022] GET /identity/v3/users/7f3bbab298794248859fc6f8b45e2e96 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:16.209453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df9cae-0979-439b-a4ca-e654cfb9f056 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:16.271221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:16.279205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:16.307700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c33aa-db9e-4c29-b998-7487fed65a86 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:16.329059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df9cae-0979-439b-a4ca-e654cfb9f056 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:16.345129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.345843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.346131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.346411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.347174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:16.347488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.348895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:16.357892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 96/197] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:16.358566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:16.373070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe50ae-e678-459c-9cc4-40674a08bdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.374618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe50ae-e678-459c-9cc4-40674a08bdfe None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.374903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe50ae-e678-459c-9cc4-40674a08bdfe None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.375179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe50ae-e678-459c-9cc4-40674a08bdfe None admin] PATH_INFO: `/v3/users/7f3bbab298794248859fc6f8b45e2e96` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.377193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe50ae-e678-459c-9cc4-40674a08bdfe None admin] RBAC: Authorizing `identity:get_user(user_id=7f3bbab298794248859fc6f8b45e2e96)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.378649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe50ae-e678-459c-9cc4-40674a08bdfe None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:16.380288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 102/198] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:16 2022] GET /identity/v3/users/7f3bbab298794248859fc6f8b45e2e96 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:16.381101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe50ae-e678-459c-9cc4-40674a08bdfe None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:16.488123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861d200-5edd-465b-8bb5-4aa92b5f08ce None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:16.511529 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe50ae-e678-459c-9cc4-40674a08bdfe None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:16.527857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e056200-adfe-4e9c-9933-374d637ca1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.528703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e056200-adfe-4e9c-9933-374d637ca1ac None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.529116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e056200-adfe-4e9c-9933-374d637ca1ac None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.529407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e056200-adfe-4e9c-9933-374d637ca1ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.538832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:16.542391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:16.551718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d12925-6d1b-465c-8a40-cd090d3b5db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.552645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12925-6d1b-465c-8a40-cd090d3b5db3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.557995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12925-6d1b-465c-8a40-cd090d3b5db3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.557995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12925-6d1b-465c-8a40-cd090d3b5db3 None admin] PATH_INFO: `/v3/users/e0c5befcce06499a8ee42cf582c22b58` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.557995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d12925-6d1b-465c-8a40-cd090d3b5db3 None admin] RBAC: Authorizing `identity:get_user(user_id=e0c5befcce06499a8ee42cf582c22b58)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.557995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d12925-6d1b-465c-8a40-cd090d3b5db3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:16.558264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 103/199] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:16 2022] GET /identity/v3/users/e0c5befcce06499a8ee42cf582c22b58 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:16.558952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d12925-6d1b-465c-8a40-cd090d3b5db3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:16.621656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e056200-adfe-4e9c-9933-374d637ca1ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.623219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e056200-adfe-4e9c-9933-374d637ca1ac None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:16.687793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d12925-6d1b-465c-8a40-cd090d3b5db3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:16.724872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbbfcd-0785-45a6-b71c-e2a6aba748e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.726070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbbfcd-0785-45a6-b71c-e2a6aba748e7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.726562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbbfcd-0785-45a6-b71c-e2a6aba748e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.727044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbbfcd-0785-45a6-b71c-e2a6aba748e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.736374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 97/200] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:16.737049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e056200-adfe-4e9c-9933-374d637ca1ac None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:16.774369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:16.800902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:16.809326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbbfcd-0785-45a6-b71c-e2a6aba748e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.811596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbbfcd-0785-45a6-b71c-e2a6aba748e7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:16.860156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e056200-adfe-4e9c-9933-374d637ca1ac None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:16.901957 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb31bc7-73b2-4456-a9e9-5a1b71971be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:16.902979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb31bc7-73b2-4456-a9e9-5a1b71971be3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:16.903491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb31bc7-73b2-4456-a9e9-5a1b71971be3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:16.903971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb31bc7-73b2-4456-a9e9-5a1b71971be3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:16.933895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 104/201] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:16.933895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbbfcd-0785-45a6-b71c-e2a6aba748e7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:16.980873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb31bc7-73b2-4456-a9e9-5a1b71971be3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:16.982755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb31bc7-73b2-4456-a9e9-5a1b71971be3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:17.016914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:17.030485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:17.035589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbbfcd-0785-45a6-b71c-e2a6aba748e7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:17.071367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb51e4-dc05-4d5f-9282-db7af21e7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:17.072403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb51e4-dc05-4d5f-9282-db7af21e7e9c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.072669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb51e4-dc05-4d5f-9282-db7af21e7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.072965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb51e4-dc05-4d5f-9282-db7af21e7e9c None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.074163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb51e4-dc05-4d5f-9282-db7af21e7e9c None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:17.075671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb51e4-dc05-4d5f-9282-db7af21e7e9c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:17.083089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 105/202] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:16 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:17.083856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb51e4-dc05-4d5f-9282-db7af21e7e9c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:17.093612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 98/203] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:17.094913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb31bc7-73b2-4456-a9e9-5a1b71971be3 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:17.191203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb51e4-dc05-4d5f-9282-db7af21e7e9c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:17.200856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb31bc7-73b2-4456-a9e9-5a1b71971be3 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:17.227247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e911b-84a6-4971-91f7-bb62f1747d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:17.228701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e911b-84a6-4971-91f7-bb62f1747d57 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.228993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e911b-84a6-4971-91f7-bb62f1747d57 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.229274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e911b-84a6-4971-91f7-bb62f1747d57 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.230479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e911b-84a6-4971-91f7-bb62f1747d57 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:17.231927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e911b-84a6-4971-91f7-bb62f1747d57 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:17.233663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 106/204] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:17 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:17.234425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e911b-84a6-4971-91f7-bb62f1747d57 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:17.240492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:17.241466 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.241955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.242488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.243647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] RBAC: Adding query filter params (role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=f03d23cb840648a38723008cbfdbbde7, user.id=7f3bbab298794248859fc6f8b45e2e96) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:17.244165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:17.245887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:17.247983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:17.252975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 99/205] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:17 2022] GET /identity/v3/role_assignments?user.id=7f3bbab298794248859fc6f8b45e2e96&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:17.254197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:17.273868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:17.361002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e911b-84a6-4971-91f7-bb62f1747d57 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:17.370288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cff711-72b2-487b-930f-ab307976e091 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:17.402912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5975f-9638-4527-9976-39391ff5beea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:17.403808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5975f-9638-4527-9976-39391ff5beea None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.404144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5975f-9638-4527-9976-39391ff5beea None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.404351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5975f-9638-4527-9976-39391ff5beea None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.405727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5975f-9638-4527-9976-39391ff5beea None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:17.407076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5975f-9638-4527-9976-39391ff5beea None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:17.409177 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 107/206] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:17 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:17.411311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068a06a-e46c-48f6-95fe-2fc9cb88f2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:17.411981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a06a-e46c-48f6-95fe-2fc9cb88f2f4 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.412269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a06a-e46c-48f6-95fe-2fc9cb88f2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.420579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a06a-e46c-48f6-95fe-2fc9cb88f2f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.420579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068a06a-e46c-48f6-95fe-2fc9cb88f2f4 None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=f03d23cb840648a38723008cbfdbbde7, user.id=7f3bbab298794248859fc6f8b45e2e96) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:17.420579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068a06a-e46c-48f6-95fe-2fc9cb88f2f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:17.420579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068a06a-e46c-48f6-95fe-2fc9cb88f2f4 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:17.422152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5975f-9638-4527-9976-39391ff5beea None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:17.423541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 100/207] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:17 2022] GET /identity/v3/role_assignments?user.id=7f3bbab298794248859fc6f8b45e2e96&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:17.537387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:17.542546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5975f-9638-4527-9976-39391ff5beea None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:17.583482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39664587-0354-475f-94e6-0d197f70cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:17.584509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39664587-0354-475f-94e6-0d197f70cc10 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.584996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39664587-0354-475f-94e6-0d197f70cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.585520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39664587-0354-475f-94e6-0d197f70cc10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.586585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39664587-0354-475f-94e6-0d197f70cc10 None admin] RBAC: Adding query filter params (role.id=062e902156a0420f882cc116e074cc88, scope.project.id=bcf793428063417f9cb53b3d0b4eea85, user.id=e0c5befcce06499a8ee42cf582c22b58) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:17.587084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39664587-0354-475f-94e6-0d197f70cc10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:17.589294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39664587-0354-475f-94e6-0d197f70cc10 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:17.599289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 108/208] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:17 2022] GET /identity/v3/role_assignments?user.id=e0c5befcce06499a8ee42cf582c22b58&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=062e902156a0420f882cc116e074cc88 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:18.572979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e60ded-6f54-4c94-b3eb-606064aa3e81 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.573486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e60ded-6f54-4c94-b3eb-606064aa3e81 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.573568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e60ded-6f54-4c94-b3eb-606064aa3e81 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.574921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 101/209] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:18.580276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fecd13-71b4-451f-877f-40db73ede74d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.580734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fecd13-71b4-451f-877f-40db73ede74d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.580918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fecd13-71b4-451f-877f-40db73ede74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.600079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-93fecd13-71b4-451f-877f-40db73ede74d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:18.601494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-93fecd13-71b4-451f-877f-40db73ede74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:18.673641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 109/210] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:18.697623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e60ded-6f54-4c94-b3eb-606064aa3e81 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:18.758134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988baeca-954f-49e9-97bf-5a2aa68f2f91 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.758589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988baeca-954f-49e9-97bf-5a2aa68f2f91 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.759023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988baeca-954f-49e9-97bf-5a2aa68f2f91 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.760743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 110/211] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:18.765713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473218-3858-4508-8e56-bf043951ddf9 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.766081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473218-3858-4508-8e56-bf043951ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.766435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473218-3858-4508-8e56-bf043951ddf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.774586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:18.784198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e6473218-3858-4508-8e56-bf043951ddf9 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:18.785557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e6473218-3858-4508-8e56-bf043951ddf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:18.828312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e60ded-6f54-4c94-b3eb-606064aa3e81 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:18.864098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4e7d-424a-4f19-a8f7-ed1941f622b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:18.865014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4e7d-424a-4f19-a8f7-ed1941f622b3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.865418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4e7d-424a-4f19-a8f7-ed1941f622b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.865772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4e7d-424a-4f19-a8f7-ed1941f622b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.872457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 111/212] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:18.880581 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a4e7d-424a-4f19-a8f7-ed1941f622b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-cb2a4e7d-424a-4f19-a8f7-ed1941f622b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:18.888482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:18.889418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:18.890222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 102/213] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:18.897066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6473218-3858-4508-8e56-bf043951ddf9 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:18.900654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4e7d-424a-4f19-a8f7-ed1941f622b3 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:18.928857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:19.015039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6473218-3858-4508-8e56-bf043951ddf9 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:19.028050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4e7d-424a-4f19-a8f7-ed1941f622b3 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:19.057509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:19.058509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.059044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.059543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.060747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:19.061034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:19.062171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:19.063797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:19.076585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966c8f5-32da-4bd8-9026-067749b6de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:19.076585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966c8f5-32da-4bd8-9026-067749b6de59 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.076585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966c8f5-32da-4bd8-9026-067749b6de59 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.076585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966c8f5-32da-4bd8-9026-067749b6de59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.076939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966c8f5-32da-4bd8-9026-067749b6de59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:19.077456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 112/214] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:19.078888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966c8f5-32da-4bd8-9026-067749b6de59 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:19.079993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:19.080437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 103/215] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:19.085835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966c8f5-32da-4bd8-9026-067749b6de59 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:19.180708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:19.198895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea190645-7331-4cd3-970c-ffeba49203c2 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:19.207140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966c8f5-32da-4bd8-9026-067749b6de59 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:19.237921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83eef-79db-423e-9fec-b3f17f4dccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:19.239156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83eef-79db-423e-9fec-b3f17f4dccfc None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.239861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83eef-79db-423e-9fec-b3f17f4dccfc None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.240461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83eef-79db-423e-9fec-b3f17f4dccfc None admin] PATH_INFO: `/v3/users/47ff5dafe81140b2a13dacfb45f9a382` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.242926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9b067-0406-42a8-aafe-1ed7b431957e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:19.243646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9b067-0406-42a8-aafe-1ed7b431957e None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.243938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9b067-0406-42a8-aafe-1ed7b431957e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.244296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9b067-0406-42a8-aafe-1ed7b431957e None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.245393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9b067-0406-42a8-aafe-1ed7b431957e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:19.246504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9b067-0406-42a8-aafe-1ed7b431957e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:19.258048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e83eef-79db-423e-9fec-b3f17f4dccfc None admin] RBAC: Authorizing `identity:get_user(user_id=47ff5dafe81140b2a13dacfb45f9a382)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:19.260493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e83eef-79db-423e-9fec-b3f17f4dccfc None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:19.267005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 113/216] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:19 2022] GET /identity/v3/users/47ff5dafe81140b2a13dacfb45f9a382 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:19.268314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83eef-79db-423e-9fec-b3f17f4dccfc None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:19.340904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 104/217] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:19 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:19.391236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83eef-79db-423e-9fec-b3f17f4dccfc None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:19.433293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a3b0a-f6be-4623-8d85-34edeefc375e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:19.434289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3b0a-f6be-4623-8d85-34edeefc375e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.434720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3b0a-f6be-4623-8d85-34edeefc375e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.435157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3b0a-f6be-4623-8d85-34edeefc375e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.449606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:19.511452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a3b0a-f6be-4623-8d85-34edeefc375e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:19.513191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a3b0a-f6be-4623-8d85-34edeefc375e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:19.636319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 114/218] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:16:19.642672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9b067-0406-42a8-aafe-1ed7b431957e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:19.669026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c50ba-2618-4c71-be16-0f2f7929e252 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.669331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c50ba-2618-4c71-be16-0f2f7929e252 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.669625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c50ba-2618-4c71-be16-0f2f7929e252 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.672341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 115/219] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:19.679407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce5a13-fa32-4913-841c-1925d27f2d6c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.679702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce5a13-fa32-4913-841c-1925d27f2d6c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.679987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce5a13-fa32-4913-841c-1925d27f2d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.704107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-efce5a13-fa32-4913-841c-1925d27f2d6c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:19.705309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-efce5a13-fa32-4913-841c-1925d27f2d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:19.721468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:19.756220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:19.799201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9b067-0406-42a8-aafe-1ed7b431957e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:19.801330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 116/220] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:19.815009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906ee88-f373-4414-91cc-dd1717debd7c None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.815315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906ee88-f373-4414-91cc-dd1717debd7c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.815585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906ee88-f373-4414-91cc-dd1717debd7c None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.817160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 117/221] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:19.824050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c279f5-b3ea-4d76-8ade-001c175ccf45 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.824740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c279f5-b3ea-4d76-8ade-001c175ccf45 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.825205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c279f5-b3ea-4d76-8ade-001c175ccf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.845654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-59c279f5-b3ea-4d76-8ade-001c175ccf45 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:19.846722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb228ea3-c52a-4227-9ab2-9347bc859bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:19.846851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-59c279f5-b3ea-4d76-8ade-001c175ccf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:19.847891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb228ea3-c52a-4227-9ab2-9347bc859bda None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:19.848358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb228ea3-c52a-4227-9ab2-9347bc859bda None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:19.848958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb228ea3-c52a-4227-9ab2-9347bc859bda None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:19.850485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb228ea3-c52a-4227-9ab2-9347bc859bda None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:19.852128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb228ea3-c52a-4227-9ab2-9347bc859bda None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:19.854641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 105/222] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:19 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:19.855439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb228ea3-c52a-4227-9ab2-9347bc859bda None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:19.917324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 118/223] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:19.918189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c279f5-b3ea-4d76-8ade-001c175ccf45 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:19.991579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb228ea3-c52a-4227-9ab2-9347bc859bda None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.006667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:20.027682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc93389-2d36-462d-b4f8-be780948c4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.028503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc93389-2d36-462d-b4f8-be780948c4e4 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.028647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c279f5-b3ea-4d76-8ade-001c175ccf45 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.028834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc93389-2d36-462d-b4f8-be780948c4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.029117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc93389-2d36-462d-b4f8-be780948c4e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.030122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:20.041131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc93389-2d36-462d-b4f8-be780948c4e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-5bc93389-2d36-462d-b4f8-be780948c4e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:20.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:20.051635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:20.052469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 106/224] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:20.053726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc93389-2d36-462d-b4f8-be780948c4e4 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.067560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.068229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.068531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.068921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.069889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] RBAC: Adding query filter params (role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=f03d23cb840648a38723008cbfdbbde7, user.id=47ff5dafe81140b2a13dacfb45f9a382) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:20.070211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:20.071683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:20.078820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 119/225] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:19 2022] GET /identity/v3/role_assignments?user.id=47ff5dafe81140b2a13dacfb45f9a382&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=64517449a51047a3b2ce1079b4912f57 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:20.085890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.161681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc93389-2d36-462d-b4f8-be780948c4e4 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.202502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d550c2b-c22b-47c8-8344-186f9705b181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.203492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d550c2b-c22b-47c8-8344-186f9705b181 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.203790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d550c2b-c22b-47c8-8344-186f9705b181 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.204075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d550c2b-c22b-47c8-8344-186f9705b181 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.217496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d550c2b-c22b-47c8-8344-186f9705b181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-1d550c2b-c22b-47c8-8344-186f9705b181 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:20.226250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:20.227211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:20.228621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 107/226] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:20.241608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d550c2b-c22b-47c8-8344-186f9705b181 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.278373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c169c79-ee5c-4308-ae05-d3eaf7966522 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.317844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.319137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.319637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.320125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.321226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:20.322418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:20.324051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:20.328555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:20.335798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 120/227] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:20.343454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.378099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:20.465333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f3120-0606-4e02-89db-af4935b317e5 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.502692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d550c2b-c22b-47c8-8344-186f9705b181 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.506205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c60ec-113e-4980-9467-7789dd61047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.507325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c60ec-113e-4980-9467-7789dd61047a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.507808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c60ec-113e-4980-9467-7789dd61047a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.508283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c60ec-113e-4980-9467-7789dd61047a None admin] PATH_INFO: `/v3/users/7f3bbab298794248859fc6f8b45e2e96` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.510786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c60ec-113e-4980-9467-7789dd61047a None admin] RBAC: Authorizing `identity:get_user(user_id=7f3bbab298794248859fc6f8b45e2e96)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:20.512446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c60ec-113e-4980-9467-7789dd61047a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:20.514616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 121/228] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/users/7f3bbab298794248859fc6f8b45e2e96 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:20.519492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c60ec-113e-4980-9467-7789dd61047a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.572965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.574047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.574752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.581164 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.582851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:20.583550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:20.586313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:20.610533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 108/229] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:20.615949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.686925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:20.695245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c60ec-113e-4980-9467-7789dd61047a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.731512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196426ae-0a36-4f32-9ebd-8541080fe361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.732298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196426ae-0a36-4f32-9ebd-8541080fe361 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.732611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196426ae-0a36-4f32-9ebd-8541080fe361 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.732892 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196426ae-0a36-4f32-9ebd-8541080fe361 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.734397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196426ae-0a36-4f32-9ebd-8541080fe361 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:20.736169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196426ae-0a36-4f32-9ebd-8541080fe361 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:20.738294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 122/230] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:20.744689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196426ae-0a36-4f32-9ebd-8541080fe361 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.851399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196426ae-0a36-4f32-9ebd-8541080fe361 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.858620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:20.879289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd801fe-1851-4894-9805-ef09e6e9fc97 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.906195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9584121-8a23-4db4-bdbf-67bc0c5f58d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.907098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584121-8a23-4db4-bdbf-67bc0c5f58d1 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.907380 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584121-8a23-4db4-bdbf-67bc0c5f58d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.907655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584121-8a23-4db4-bdbf-67bc0c5f58d1 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7/users/7f3bbab298794248859fc6f8b45e2e96/roles/96431666cb824a7087dbd9f0e30c0c80` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.910711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9584121-8a23-4db4-bdbf-67bc0c5f58d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f03d23cb840648a38723008cbfdbbde7, user_id=7f3bbab298794248859fc6f8b45e2e96, role_id=96431666cb824a7087dbd9f0e30c0c80)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:20.912299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9584121-8a23-4db4-bdbf-67bc0c5f58d1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:20.942136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd04dce-3ba6-4845-8bf3-f8d3d67571e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.943023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd04dce-3ba6-4845-8bf3-f8d3d67571e7 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.943312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd04dce-3ba6-4845-8bf3-f8d3d67571e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.943591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd04dce-3ba6-4845-8bf3-f8d3d67571e7 None admin] PATH_INFO: `/v3/users/7f3bbab298794248859fc6f8b45e2e96` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.944829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd04dce-3ba6-4845-8bf3-f8d3d67571e7 None admin] RBAC: Authorizing `identity:get_user(user_id=7f3bbab298794248859fc6f8b45e2e96)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:20.946287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd04dce-3ba6-4845-8bf3-f8d3d67571e7 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:20.948391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 109/231] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/users/7f3bbab298794248859fc6f8b45e2e96 => generated 283 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:20.953427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd04dce-3ba6-4845-8bf3-f8d3d67571e7 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.998974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 123/232] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:20 2022] PUT /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7/users/7f3bbab298794248859fc6f8b45e2e96/roles/96431666cb824a7087dbd9f0e30c0c80 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:21.067664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd04dce-3ba6-4845-8bf3-f8d3d67571e7 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:21.105939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218d0fc-4041-41b7-8e86-fced3305dc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:21.106827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218d0fc-4041-41b7-8e86-fced3305dc23 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:21.107111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218d0fc-4041-41b7-8e86-fced3305dc23 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:21.107390 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218d0fc-4041-41b7-8e86-fced3305dc23 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:21.108755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218d0fc-4041-41b7-8e86-fced3305dc23 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:21.110329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218d0fc-4041-41b7-8e86-fced3305dc23 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:21.112402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 110/233] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:21.118036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9584121-8a23-4db4-bdbf-67bc0c5f58d1 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:21.164237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:21.254927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9584121-8a23-4db4-bdbf-67bc0c5f58d1 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:21.292235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8236110-7314-4de1-b9d0-8c28de24b138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:21.293171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236110-7314-4de1-b9d0-8c28de24b138 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:21.293474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236110-7314-4de1-b9d0-8c28de24b138 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:21.293734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236110-7314-4de1-b9d0-8c28de24b138 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7/users/7f3bbab298794248859fc6f8b45e2e96/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:21.295700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8236110-7314-4de1-b9d0-8c28de24b138 None admin] RBAC: Authorizing `identity:create_grant(project_id=f03d23cb840648a38723008cbfdbbde7, user_id=7f3bbab298794248859fc6f8b45e2e96, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:21.297185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8236110-7314-4de1-b9d0-8c28de24b138 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:21.397988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 124/234] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:21 2022] PUT /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7/users/7f3bbab298794248859fc6f8b45e2e96/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:22.092328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b26f6-18ed-4fc8-b8fe-f2d08b5fab4d None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.093077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b26f6-18ed-4fc8-b8fe-f2d08b5fab4d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.093077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b26f6-18ed-4fc8-b8fe-f2d08b5fab4d None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.094497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 111/235] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:22.099875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1074e84-916f-46f8-935a-613c5ce8ad78 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.100365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1074e84-916f-46f8-935a-613c5ce8ad78 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.100563 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1074e84-916f-46f8-935a-613c5ce8ad78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.121493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c1074e84-916f-46f8-935a-613c5ce8ad78 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:22.122683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c1074e84-916f-46f8-935a-613c5ce8ad78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:22.125963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc28589-66ff-4750-b5bd-7626374ab77c None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.126256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc28589-66ff-4750-b5bd-7626374ab77c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.126543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc28589-66ff-4750-b5bd-7626374ab77c None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.127904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 112/236] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:22.132890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aedb17-a651-4277-bf54-033db8e4f9c6 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.133179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aedb17-a651-4277-bf54-033db8e4f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.133459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aedb17-a651-4277-bf54-033db8e4f9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.160363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-17aedb17-a651-4277-bf54-033db8e4f9c6 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:22.161578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-17aedb17-a651-4277-bf54-033db8e4f9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:22.210221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:22.244183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 125/237] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:22.276994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 113/238] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:22.289646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1074e84-916f-46f8-935a-613c5ce8ad78 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:22.305883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aedb17-a651-4277-bf54-033db8e4f9c6 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:22.312866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:22.401774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1074e84-916f-46f8-935a-613c5ce8ad78 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:22.442459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea216218-3396-4fb1-8801-207fc49bc191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:22.443303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216218-3396-4fb1-8801-207fc49bc191 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.443587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216218-3396-4fb1-8801-207fc49bc191 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.443864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216218-3396-4fb1-8801-207fc49bc191 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.445226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea216218-3396-4fb1-8801-207fc49bc191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:22.447449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea216218-3396-4fb1-8801-207fc49bc191 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:22.449609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 126/239] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:22.454973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea216218-3396-4fb1-8801-207fc49bc191 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:22.514296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aedb17-a651-4277-bf54-033db8e4f9c6 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:22.553139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03608-6405-4a59-b837-71d1deec2163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:22.554214 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03608-6405-4a59-b837-71d1deec2163 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.554501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03608-6405-4a59-b837-71d1deec2163 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.554782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03608-6405-4a59-b837-71d1deec2163 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.567905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d03608-6405-4a59-b837-71d1deec2163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-c2d03608-6405-4a59-b837-71d1deec2163 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:22.583908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:22.585136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:22.586186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 114/240] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:22.586864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43ad61-4ff1-468d-8c4b-e11e9ead36fc None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.586864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43ad61-4ff1-468d-8c4b-e11e9ead36fc None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.586983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43ad61-4ff1-468d-8c4b-e11e9ead36fc None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.598242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 115/241] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:22 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:22.599228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43ad61-4ff1-468d-8c4b-e11e9ead36fc None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:22.624991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:22.625669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:22.637010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea216218-3396-4fb1-8801-207fc49bc191 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:22.673972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828ca35-8111-4987-9a38-551e820e05e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:22.674705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828ca35-8111-4987-9a38-551e820e05e1 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.675093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828ca35-8111-4987-9a38-551e820e05e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.675495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828ca35-8111-4987-9a38-551e820e05e1 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.676305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828ca35-8111-4987-9a38-551e820e05e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:22.678301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828ca35-8111-4987-9a38-551e820e05e1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:22.715073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43ad61-4ff1-468d-8c4b-e11e9ead36fc None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:22.750065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:22.750904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.751333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.751750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.752696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:22.753146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:22.754644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:22.758267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 127/242] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:22 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:22.759817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d673d2-2d94-4fcb-b49b-1698866677dc None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.760215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d673d2-2d94-4fcb-b49b-1698866677dc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.760912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d673d2-2d94-4fcb-b49b-1698866677dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.764621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 116/243] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:22.770426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:22.781196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c5d673d2-2d94-4fcb-b49b-1698866677dc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:22.782655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c5d673d2-2d94-4fcb-b49b-1698866677dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:22.866895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 128/244] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:22.874718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:22.893125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28b1ec-a4f2-4d60-aec1-ee7d7024acf4 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:22.893362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d673d2-2d94-4fcb-b49b-1698866677dc None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:22.939200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413a872-f9a1-46bf-b803-7e8d78117017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:22.940359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413a872-f9a1-46bf-b803-7e8d78117017 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:22.942016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413a872-f9a1-46bf-b803-7e8d78117017 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:22.942457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413a872-f9a1-46bf-b803-7e8d78117017 None admin] PATH_INFO: `/v3/users/d2df74a2d11543b9af02b10682ee058b` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:22.960419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413a872-f9a1-46bf-b803-7e8d78117017 None admin] RBAC: Authorizing `identity:get_user(user_id=d2df74a2d11543b9af02b10682ee058b)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:22.962205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413a872-f9a1-46bf-b803-7e8d78117017 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:22.965034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 117/245] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:22 2022] GET /identity/v3/users/d2df74a2d11543b9af02b10682ee058b => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:22.970551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413a872-f9a1-46bf-b803-7e8d78117017 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:23.010908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:23.034846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d673d2-2d94-4fcb-b49b-1698866677dc None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:23.075233 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1109eb9-9978-419e-b335-45a040887682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:23.076143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109eb9-9978-419e-b335-45a040887682 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.076479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109eb9-9978-419e-b335-45a040887682 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.076864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109eb9-9978-419e-b335-45a040887682 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.083682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413a872-f9a1-46bf-b803-7e8d78117017 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:23.089012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1109eb9-9978-419e-b335-45a040887682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-b1109eb9-9978-419e-b335-45a040887682 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:23.097053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:23.098154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:23.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 129/246] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:23.104514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1109eb9-9978-419e-b335-45a040887682 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:23.124000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0c84-a4b8-45bc-92e3-159f59a37f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:23.125260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0c84-a4b8-45bc-92e3-159f59a37f20 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.125698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0c84-a4b8-45bc-92e3-159f59a37f20 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.126165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0c84-a4b8-45bc-92e3-159f59a37f20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.139733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:23.204451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0c84-a4b8-45bc-92e3-159f59a37f20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:23.206257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0c84-a4b8-45bc-92e3-159f59a37f20 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:23.219348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1109eb9-9978-419e-b335-45a040887682 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:23.258811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:23.260213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.260598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.261088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.261941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:23.262349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:23.263990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:23.273521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 130/247] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:23.273786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:23.280142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:23.319709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 118/248] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:16:23.326720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0c84-a4b8-45bc-92e3-159f59a37f20 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:23.371057 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:23.395966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ef983-f4c8-416d-87ef-f82ff4dbbdf9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:23.437228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0c84-a4b8-45bc-92e3-159f59a37f20 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:23.438123 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc360c0a-f4ec-4da9-88dc-157c63447001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:23.439215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc360c0a-f4ec-4da9-88dc-157c63447001 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.439596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc360c0a-f4ec-4da9-88dc-157c63447001 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.439969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc360c0a-f4ec-4da9-88dc-157c63447001 None admin] PATH_INFO: `/v3/users/47ff5dafe81140b2a13dacfb45f9a382` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.441872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc360c0a-f4ec-4da9-88dc-157c63447001 None admin] RBAC: Authorizing `identity:get_user(user_id=47ff5dafe81140b2a13dacfb45f9a382)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:23.443432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc360c0a-f4ec-4da9-88dc-157c63447001 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:23.445634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 131/249] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:23 2022] GET /identity/v3/users/47ff5dafe81140b2a13dacfb45f9a382 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:23.451653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc360c0a-f4ec-4da9-88dc-157c63447001 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:23.479663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29665d2-d539-4c97-8712-41002da68246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:23.480810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29665d2-d539-4c97-8712-41002da68246 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.481238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29665d2-d539-4c97-8712-41002da68246 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.481665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29665d2-d539-4c97-8712-41002da68246 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.483015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29665d2-d539-4c97-8712-41002da68246 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:23.484703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29665d2-d539-4c97-8712-41002da68246 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:23.486580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 119/250] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:23 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:23.493887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29665d2-d539-4c97-8712-41002da68246 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:23.540913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:23.567291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc360c0a-f4ec-4da9-88dc-157c63447001 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:23.615764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beec57-1362-4604-908f-ecc566f0b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:23.617493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beec57-1362-4604-908f-ecc566f0b194 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.618164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beec57-1362-4604-908f-ecc566f0b194 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.619435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beec57-1362-4604-908f-ecc566f0b194 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.622063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beec57-1362-4604-908f-ecc566f0b194 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:23.623928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beec57-1362-4604-908f-ecc566f0b194 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:23.626010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 132/251] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:23 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:23.626870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29665d2-d539-4c97-8712-41002da68246 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:23.631977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beec57-1362-4604-908f-ecc566f0b194 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:23.639006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:23.662956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01d000-5535-413d-8b38-bab0a2598f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:23.667614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01d000-5535-413d-8b38-bab0a2598f9b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.667614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01d000-5535-413d-8b38-bab0a2598f9b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.667614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01d000-5535-413d-8b38-bab0a2598f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.667614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01d000-5535-413d-8b38-bab0a2598f9b None admin] RBAC: Adding query filter params (role.id=062e902156a0420f882cc116e074cc88, scope.project.id=f03d23cb840648a38723008cbfdbbde7, user.id=d2df74a2d11543b9af02b10682ee058b) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:23.667614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01d000-5535-413d-8b38-bab0a2598f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:23.667614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01d000-5535-413d-8b38-bab0a2598f9b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:23.675540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 120/252] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:23 2022] GET /identity/v3/role_assignments?user.id=d2df74a2d11543b9af02b10682ee058b&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=062e902156a0420f882cc116e074cc88 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:23.689637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615fc64-290e-4948-b788-d9c306d4c10f None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.696540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615fc64-290e-4948-b788-d9c306d4c10f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.696540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615fc64-290e-4948-b788-d9c306d4c10f None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.696540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 121/253] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:23.697627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a86d1e-48c2-4e12-b4f7-7e2259e97e8e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.698098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a86d1e-48c2-4e12-b4f7-7e2259e97e8e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.698552 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a86d1e-48c2-4e12-b4f7-7e2259e97e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.724104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f2a86d1e-48c2-4e12-b4f7-7e2259e97e8e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:23.725698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f2a86d1e-48c2-4e12-b4f7-7e2259e97e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:23.752788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beec57-1362-4604-908f-ecc566f0b194 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:23.778538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:23.793364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a6e8b-07c7-4fa2-9e72-65385f234285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:23.795014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a6e8b-07c7-4fa2-9e72-65385f234285 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:23.795492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a6e8b-07c7-4fa2-9e72-65385f234285 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:23.796040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a6e8b-07c7-4fa2-9e72-65385f234285 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7/users/47ff5dafe81140b2a13dacfb45f9a382/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:23.805590 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a6e8b-07c7-4fa2-9e72-65385f234285 None admin] RBAC: Authorizing `identity:create_grant(project_id=f03d23cb840648a38723008cbfdbbde7, user_id=47ff5dafe81140b2a13dacfb45f9a382, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:23.807833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a6e8b-07c7-4fa2-9e72-65385f234285 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:23.888838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 122/254] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:23.897248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 133/255] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:23 2022] PUT /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7/users/47ff5dafe81140b2a13dacfb45f9a382/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:16:23.933437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a86d1e-48c2-4e12-b4f7-7e2259e97e8e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:24.079209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a86d1e-48c2-4e12-b4f7-7e2259e97e8e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:24.085823 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:24.136838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7d0a1-f284-4d44-9df7-de9c14546195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:24.137683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7d0a1-f284-4d44-9df7-de9c14546195 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:24.137964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7d0a1-f284-4d44-9df7-de9c14546195 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:24.138239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7d0a1-f284-4d44-9df7-de9c14546195 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:24.157375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7d0a1-f284-4d44-9df7-de9c14546195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-28a7d0a1-f284-4d44-9df7-de9c14546195 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:24.165941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:24.166943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:24.167757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 123/256] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:24.173909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a6e8b-07c7-4fa2-9e72-65385f234285 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:24.248895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01be38-503d-4b06-b974-c09c4565a7df None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:24.249224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01be38-503d-4b06-b974-c09c4565a7df None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:24.249533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01be38-503d-4b06-b974-c09c4565a7df None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:24.252198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 124/257] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:24.256197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4ca7e-3117-4188-acfb-c855c9754908 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:24.256627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4ca7e-3117-4188-acfb-c855c9754908 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:24.256913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4ca7e-3117-4188-acfb-c855c9754908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:24.275181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-69c4ca7e-3117-4188-acfb-c855c9754908 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:24.276384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-69c4ca7e-3117-4188-acfb-c855c9754908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:24.306642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a6e8b-07c7-4fa2-9e72-65385f234285 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:24.346551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:24.347279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:24.347766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:24.348102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:24.348949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:24.349266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:24.350943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:24.367319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 134/258] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:24.367577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 125/259] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:24.371475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:24.410862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4ca7e-3117-4188-acfb-c855c9754908 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:24.435145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:24.532569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb3f42-9f88-4c62-9cb4-d9202a706e79 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:24.554404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:24.562256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4ca7e-3117-4188-acfb-c855c9754908 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:24.605541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afdc7a-47da-4f45-b180-6d8297e1c4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:24.606276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afdc7a-47da-4f45-b180-6d8297e1c4b7 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:24.606565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afdc7a-47da-4f45-b180-6d8297e1c4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:24.606899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afdc7a-47da-4f45-b180-6d8297e1c4b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:24.623707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297df039-d6d7-48e8-b3f8-6f47e34f3f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:24.624667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297df039-d6d7-48e8-b3f8-6f47e34f3f0b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:24.624947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297df039-d6d7-48e8-b3f8-6f47e34f3f0b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:24.625222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297df039-d6d7-48e8-b3f8-6f47e34f3f0b None admin] PATH_INFO: `/v3/users/7f3bbab298794248859fc6f8b45e2e96` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:24.626685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297df039-d6d7-48e8-b3f8-6f47e34f3f0b None admin] RBAC: Authorizing `identity:get_user(user_id=7f3bbab298794248859fc6f8b45e2e96)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:24.627976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afdc7a-47da-4f45-b180-6d8297e1c4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:24.631510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297df039-d6d7-48e8-b3f8-6f47e34f3f0b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:24.633227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 135/260] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:24 2022] GET /identity/v3/users/7f3bbab298794248859fc6f8b45e2e96 => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:24.638748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297df039-d6d7-48e8-b3f8-6f47e34f3f0b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-47afdc7a-47da-4f45-b180-6d8297e1c4b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:24.639960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:24.640972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:24.642514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 126/261] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:24.647104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afdc7a-47da-4f45-b180-6d8297e1c4b7 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:24.822927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297df039-d6d7-48e8-b3f8-6f47e34f3f0b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:24.834227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afdc7a-47da-4f45-b180-6d8297e1c4b7 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:24.841195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:24.860748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27d8ae-7c5d-45ec-b7a7-1cafafdfafc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:24.861375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27d8ae-7c5d-45ec-b7a7-1cafafdfafc7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:24.861660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27d8ae-7c5d-45ec-b7a7-1cafafdfafc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:24.861989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27d8ae-7c5d-45ec-b7a7-1cafafdfafc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:24.894102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:24.894775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:24.895060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:24.895339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:24.896098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:24.896407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:24.897827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:24.912355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 127/262] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:24.930462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:24.939920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27d8ae-7c5d-45ec-b7a7-1cafafdfafc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:24.941534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27d8ae-7c5d-45ec-b7a7-1cafafdfafc7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:24.967612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:25.055879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 136/263] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:16:25.057160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27d8ae-7c5d-45ec-b7a7-1cafafdfafc7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:25.076166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:25.119035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f4d8-bfdb-4121-889b-bd9860b757c9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:25.171976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a812a-005b-4a2a-97f9-01554582e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:25.176765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27d8ae-7c5d-45ec-b7a7-1cafafdfafc7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:25.178163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a812a-005b-4a2a-97f9-01554582e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:25.178666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a812a-005b-4a2a-97f9-01554582e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:25.179172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a812a-005b-4a2a-97f9-01554582e0c0 None admin] PATH_INFO: `/v3/users/7f3bbab298794248859fc6f8b45e2e96` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:25.180672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a812a-005b-4a2a-97f9-01554582e0c0 None admin] RBAC: Authorizing `identity:get_user(user_id=7f3bbab298794248859fc6f8b45e2e96)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:25.182386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a812a-005b-4a2a-97f9-01554582e0c0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:25.184608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 128/264] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:24 2022] GET /identity/v3/users/7f3bbab298794248859fc6f8b45e2e96 => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:25.191305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a812a-005b-4a2a-97f9-01554582e0c0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:25.223209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6364a-9b69-47bb-99b2-e15488776178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:25.224146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6364a-9b69-47bb-99b2-e15488776178 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:25.224437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6364a-9b69-47bb-99b2-e15488776178 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:25.224747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6364a-9b69-47bb-99b2-e15488776178 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:25.228233 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6364a-9b69-47bb-99b2-e15488776178 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:25.229855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6364a-9b69-47bb-99b2-e15488776178 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:25.231580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 137/265] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:25 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:25.237949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6364a-9b69-47bb-99b2-e15488776178 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:25.354158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:25.388268 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6364a-9b69-47bb-99b2-e15488776178 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:25.402294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:25.408487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:25.409613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:25.410105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:25.410585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:25.411681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] RBAC: Adding query filter params (role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=f03d23cb840648a38723008cbfdbbde7, user.id=7f3bbab298794248859fc6f8b45e2e96) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:25.412198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:25.414128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:25.425064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 138/266] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:25 2022] GET /identity/v3/role_assignments?user.id=7f3bbab298794248859fc6f8b45e2e96&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:25.453100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a812a-005b-4a2a-97f9-01554582e0c0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:25.504345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ebf4-ed28-4940-a307-27b7b8a47998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:25.505067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ebf4-ed28-4940-a307-27b7b8a47998 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:25.505346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ebf4-ed28-4940-a307-27b7b8a47998 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:25.505628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ebf4-ed28-4940-a307-27b7b8a47998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:25.623630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04ebf4-ed28-4940-a307-27b7b8a47998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:25.625649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04ebf4-ed28-4940-a307-27b7b8a47998 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:25.741294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:25.765638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 129/267] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:25.771264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:25.889104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be225b-4fef-4d3b-84a4-0c22e6bcf5ad None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:25.924625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c97c36-22a4-4b33-a50f-bf9df978d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:25.925857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c97c36-22a4-4b33-a50f-bf9df978d656 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:25.926416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c97c36-22a4-4b33-a50f-bf9df978d656 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:25.926903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c97c36-22a4-4b33-a50f-bf9df978d656 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:25.928337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c97c36-22a4-4b33-a50f-bf9df978d656 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:25.930084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c97c36-22a4-4b33-a50f-bf9df978d656 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:25.932569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 139/268] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:25 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:25.938519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ebf4-ed28-4940-a307-27b7b8a47998 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:26.064637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ebf4-ed28-4940-a307-27b7b8a47998 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:26.108006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d0e60-e528-40eb-8c66-62a2f344cbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:26.108711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d0e60-e528-40eb-8c66-62a2f344cbe1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:26.109000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d0e60-e528-40eb-8c66-62a2f344cbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:26.109274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d0e60-e528-40eb-8c66-62a2f344cbe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:26.110182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d0e60-e528-40eb-8c66-62a2f344cbe1 None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=f03d23cb840648a38723008cbfdbbde7, user.id=7f3bbab298794248859fc6f8b45e2e96) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:26.110501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d0e60-e528-40eb-8c66-62a2f344cbe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:26.111953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d0e60-e528-40eb-8c66-62a2f344cbe1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:26.119100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 130/269] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:25 2022] GET /identity/v3/role_assignments?user.id=7f3bbab298794248859fc6f8b45e2e96&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:26.672348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6927ce-f5aa-482a-a479-77a01abf426f None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:26.673299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6927ce-f5aa-482a-a479-77a01abf426f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:26.673299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6927ce-f5aa-482a-a479-77a01abf426f None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:26.674785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 140/270] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:26.680426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93aaa-20b4-40a5-bb4e-be18be49119d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:26.680906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93aaa-20b4-40a5-bb4e-be18be49119d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:26.681120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93aaa-20b4-40a5-bb4e-be18be49119d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:26.693915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b126-8c2c-4539-9eb0-a1cfc821566f None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:26.694526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b126-8c2c-4539-9eb0-a1cfc821566f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:26.695008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b126-8c2c-4539-9eb0-a1cfc821566f None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:26.697547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 141/271] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:26.701211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-40b93aaa-20b4-40a5-bb4e-be18be49119d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:26.702384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5418f-3c44-499b-b3e1-c0ca7e515947 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:26.702528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-40b93aaa-20b4-40a5-bb4e-be18be49119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:26.702941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5418f-3c44-499b-b3e1-c0ca7e515947 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:26.703364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5418f-3c44-499b-b3e1-c0ca7e515947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:26.724849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-27b5418f-3c44-499b-b3e1-c0ca7e515947 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:26.726286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-27b5418f-3c44-499b-b3e1-c0ca7e515947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:26.741979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:26.779554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:26.815791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 131/272] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:26.828666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 142/273] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:26.839597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b93aaa-20b4-40a5-bb4e-be18be49119d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:26.844929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5418f-3c44-499b-b3e1-c0ca7e515947 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:26.946848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b93aaa-20b4-40a5-bb4e-be18be49119d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:26.958290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5418f-3c44-499b-b3e1-c0ca7e515947 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:26.982671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5b0f-90ca-4c24-99e2-340512394d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:26.983463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5b0f-90ca-4c24-99e2-340512394d20 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:26.983747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5b0f-90ca-4c24-99e2-340512394d20 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:26.984023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5b0f-90ca-4c24-99e2-340512394d20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:26.994334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0342b3-5cce-4b83-8188-475565675410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:26.994973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f5b0f-90ca-4c24-99e2-340512394d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:26.995157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0342b3-5cce-4b83-8188-475565675410 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:26.995441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0342b3-5cce-4b83-8188-475565675410 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:26.995717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0342b3-5cce-4b83-8188-475565675410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:26.996922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0342b3-5cce-4b83-8188-475565675410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:26.998388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0342b3-5cce-4b83-8188-475565675410 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:26.999986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 143/274] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:26.999986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29bbfc-4a7c-4179-a2d2-42edd41cea5c None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:26.999986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29bbfc-4a7c-4179-a2d2-42edd41cea5c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:26.999986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29bbfc-4a7c-4179-a2d2-42edd41cea5c None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:26.999986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 144/275] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-1b1f5b0f-90ca-4c24-99e2-340512394d20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:27.004982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:27.005826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:27.006656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:27.007346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ebe4f-ec69-4ac4-9b9b-f6abbfa0ce99 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.007346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ebe4f-ec69-4ac4-9b9b-f6abbfa0ce99 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.007346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ebe4f-ec69-4ac4-9b9b-f6abbfa0ce99 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.007525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 132/276] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:27.007600 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 145/277] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:27.007668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5b0f-90ca-4c24-99e2-340512394d20 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.011483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0441e14-5ac7-43cd-a72f-c611f49aa0d2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.011771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0441e14-5ac7-43cd-a72f-c611f49aa0d2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.012047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0441e14-5ac7-43cd-a72f-c611f49aa0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.027789 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c0441e14-5ac7-43cd-a72f-c611f49aa0d2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:27.029010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c0441e14-5ac7-43cd-a72f-c611f49aa0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:27.037992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:27.049274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:27.093794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 146/278] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:27.095096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fcf1b-0665-48d9-bfc9-cbe7c7391b71 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.095321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fcf1b-0665-48d9-bfc9-cbe7c7391b71 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.095599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fcf1b-0665-48d9-bfc9-cbe7c7391b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.111436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f01fcf1b-0665-48d9-bfc9-cbe7c7391b71 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:27.112625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f01fcf1b-0665-48d9-bfc9-cbe7c7391b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:27.121975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5b0f-90ca-4c24-99e2-340512394d20 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:27.155933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68485d-6cac-42b6-815f-cd5f97c60689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:27.156623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68485d-6cac-42b6-815f-cd5f97c60689 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.156920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68485d-6cac-42b6-815f-cd5f97c60689 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.157195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68485d-6cac-42b6-815f-cd5f97c60689 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.157900 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68485d-6cac-42b6-815f-cd5f97c60689 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:27.159359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68485d-6cac-42b6-815f-cd5f97c60689 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:27.179433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 147/279] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:27.180307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fcf1b-0665-48d9-bfc9-cbe7c7391b71 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.251801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:27.253149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 133/280] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:27 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:27.254121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68485d-6cac-42b6-815f-cd5f97c60689 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.284859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fcf1b-0665-48d9-bfc9-cbe7c7391b71 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:27.326348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:27.327610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.328085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.328569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.329747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:27.330251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:27.332751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:27.342714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 148/281] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:27.343849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:27.379779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68485d-6cac-42b6-815f-cd5f97c60689 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:27.419855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f68c2-cd36-4d22-aac8-5fe904b8857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:27.420703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f68c2-cd36-4d22-aac8-5fe904b8857e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.421822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f68c2-cd36-4d22-aac8-5fe904b8857e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.422104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f68c2-cd36-4d22-aac8-5fe904b8857e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.434668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f68c2-cd36-4d22-aac8-5fe904b8857e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3d0f68c2-cd36-4d22-aac8-5fe904b8857e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:27.442895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:27.443814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:27.444679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:27.453568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 134/282] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:27.455839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f68c2-cd36-4d22-aac8-5fe904b8857e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.461361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf32d5f-e161-4144-a90d-c11f0352171f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:27.500081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add6bdb-ddad-40a6-8c6a-684902e02631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:27.500928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add6bdb-ddad-40a6-8c6a-684902e02631 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.501214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add6bdb-ddad-40a6-8c6a-684902e02631 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.501497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add6bdb-ddad-40a6-8c6a-684902e02631 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.505928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:27.516698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add6bdb-ddad-40a6-8c6a-684902e02631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-8add6bdb-ddad-40a6-8c6a-684902e02631 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:27.524646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:27.525605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:27.526463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 149/283] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:27.527161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add6bdb-ddad-40a6-8c6a-684902e02631 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.589906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f68c2-cd36-4d22-aac8-5fe904b8857e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:27.637221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb8454-7d59-48ad-b908-c5f155549569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:27.638577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb8454-7d59-48ad-b908-c5f155549569 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.638878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb8454-7d59-48ad-b908-c5f155549569 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.639170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb8454-7d59-48ad-b908-c5f155549569 None admin] PATH_INFO: `/v3/users/57095dd3af49487fa4eccb349a66a3be` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.661767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb8454-7d59-48ad-b908-c5f155549569 None admin] RBAC: Authorizing `identity:get_user(user_id=57095dd3af49487fa4eccb349a66a3be)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:27.663625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb8454-7d59-48ad-b908-c5f155549569 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:27.667295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 135/284] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/users/57095dd3af49487fa4eccb349a66a3be => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:27.672969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb8454-7d59-48ad-b908-c5f155549569 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.673106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add6bdb-ddad-40a6-8c6a-684902e02631 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:27.697485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:27.717932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:27.718608 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.726475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.726475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.726475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:27.726475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:27.726475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:27.734329 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 150/285] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:27.735781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.835927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:27.846153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb8454-7d59-48ad-b908-c5f155549569 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:27.891438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d657d5-c6f4-4d6c-84ff-c43b11329277 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:27.892365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:27.893339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.893635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.893917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.895606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:27.895932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:27.897330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:27.913278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 136/286] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:27.914831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:27.929559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:27.930266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:27.930561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:27.931376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:27.932216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] RBAC: Adding query filter params (role.id=64517449a51047a3b2ce1079b4912f57, user.id=57095dd3af49487fa4eccb349a66a3be, scope.system=all) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:27.932546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:27.934026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:27.939746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 151/287] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/role_assignments?user.id=57095dd3af49487fa4eccb349a66a3be&scope.system=all&role.id=64517449a51047a3b2ce1079b4912f57 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:27.940579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:28.018704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:28.044643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b20f-8941-424e-944b-6a2753d31b90 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:28.069363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cfb7-3678-4700-b719-248fef06ca8e None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:28.085647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd17507-ec65-4098-b46c-44222e584768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:28.086534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd17507-ec65-4098-b46c-44222e584768 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:28.086816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd17507-ec65-4098-b46c-44222e584768 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:28.087133 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd17507-ec65-4098-b46c-44222e584768 None admin] PATH_INFO: `/v3/users/47ff5dafe81140b2a13dacfb45f9a382` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:28.088410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd17507-ec65-4098-b46c-44222e584768 None admin] RBAC: Authorizing `identity:get_user(user_id=47ff5dafe81140b2a13dacfb45f9a382)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:28.089944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd17507-ec65-4098-b46c-44222e584768 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:28.091914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 137/288] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/users/47ff5dafe81140b2a13dacfb45f9a382 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:28.097416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd17507-ec65-4098-b46c-44222e584768 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:28.097695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:28.113595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c32ef8-b4af-42c8-80ed-07ccb870f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:28.114493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c32ef8-b4af-42c8-80ed-07ccb870f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:28.114780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c32ef8-b4af-42c8-80ed-07ccb870f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:28.115057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c32ef8-b4af-42c8-80ed-07ccb870f6f0 None admin] PATH_INFO: `/v3/users/d2df74a2d11543b9af02b10682ee058b` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:28.121998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c32ef8-b4af-42c8-80ed-07ccb870f6f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d2df74a2d11543b9af02b10682ee058b)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:28.123511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c32ef8-b4af-42c8-80ed-07ccb870f6f0 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:28.125299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 152/289] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:27 2022] GET /identity/v3/users/d2df74a2d11543b9af02b10682ee058b => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:28.135192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c32ef8-b4af-42c8-80ed-07ccb870f6f0 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:28.292079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd17507-ec65-4098-b46c-44222e584768 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:28.323243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c32ef8-b4af-42c8-80ed-07ccb870f6f0 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:28.347125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c4e0e-d306-4a4b-aea6-258faec034de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:28.348087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c4e0e-d306-4a4b-aea6-258faec034de None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:28.348380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c4e0e-d306-4a4b-aea6-258faec034de None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:28.348692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c4e0e-d306-4a4b-aea6-258faec034de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:28.361988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:28.424131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c4e0e-d306-4a4b-aea6-258faec034de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:28.425742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c4e0e-d306-4a4b-aea6-258faec034de None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:28.441762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33919d-eb0b-498e-8bed-5959a5a728b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:28.442645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33919d-eb0b-498e-8bed-5959a5a728b8 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:28.442928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33919d-eb0b-498e-8bed-5959a5a728b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:28.443205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33919d-eb0b-498e-8bed-5959a5a728b8 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:28.449238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33919d-eb0b-498e-8bed-5959a5a728b8 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:28.450683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33919d-eb0b-498e-8bed-5959a5a728b8 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:28.452782 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 153/290] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:28 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:28.458084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33919d-eb0b-498e-8bed-5959a5a728b8 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:28.535574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 138/291] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:28.546158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c4e0e-d306-4a4b-aea6-258faec034de None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:28.599047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:28.627857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:28.685094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33919d-eb0b-498e-8bed-5959a5a728b8 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:28.706156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c4e0e-d306-4a4b-aea6-258faec034de None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:28.796076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6000692-050f-43fe-923f-b5a5fed62056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:28.797066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6000692-050f-43fe-923f-b5a5fed62056 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:28.797302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6000692-050f-43fe-923f-b5a5fed62056 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:28.797584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6000692-050f-43fe-923f-b5a5fed62056 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7/users/d2df74a2d11543b9af02b10682ee058b/roles/062e902156a0420f882cc116e074cc88` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:28.802261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6000692-050f-43fe-923f-b5a5fed62056 None admin] RBAC: Authorizing `identity:create_grant(project_id=f03d23cb840648a38723008cbfdbbde7, user_id=d2df74a2d11543b9af02b10682ee058b, role_id=062e902156a0420f882cc116e074cc88)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:28.803934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6000692-050f-43fe-923f-b5a5fed62056 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:28.815126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766077e7-e152-4884-af4c-da209abd3cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:28.816081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766077e7-e152-4884-af4c-da209abd3cfd None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:28.816373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766077e7-e152-4884-af4c-da209abd3cfd None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:28.816773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766077e7-e152-4884-af4c-da209abd3cfd None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:28.817981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766077e7-e152-4884-af4c-da209abd3cfd None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:28.819574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766077e7-e152-4884-af4c-da209abd3cfd None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:28.822905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 139/292] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:28 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:28.828689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766077e7-e152-4884-af4c-da209abd3cfd None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:28.891731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 154/293] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:28 2022] PUT /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7/users/d2df74a2d11543b9af02b10682ee058b/roles/062e902156a0420f882cc116e074cc88 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:28.936038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766077e7-e152-4884-af4c-da209abd3cfd None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:28.948341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:28.972320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790e004-be1f-4800-8477-628aef5cee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:28.973103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790e004-be1f-4800-8477-628aef5cee1f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:28.973389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790e004-be1f-4800-8477-628aef5cee1f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:28.973670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790e004-be1f-4800-8477-628aef5cee1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:28.974523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790e004-be1f-4800-8477-628aef5cee1f None admin] RBAC: Adding query filter params (role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=f03d23cb840648a38723008cbfdbbde7, user.id=47ff5dafe81140b2a13dacfb45f9a382) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:28.974836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790e004-be1f-4800-8477-628aef5cee1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:28.976302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790e004-be1f-4800-8477-628aef5cee1f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:28.983934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 140/294] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:28 2022] GET /identity/v3/role_assignments?user.id=47ff5dafe81140b2a13dacfb45f9a382&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=64517449a51047a3b2ce1079b4912f57 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:29.684338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1091a8-38f6-4f0c-8373-e5f7133b3bf6 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:29.684681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1091a8-38f6-4f0c-8373-e5f7133b3bf6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:29.684940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1091a8-38f6-4f0c-8373-e5f7133b3bf6 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:29.686314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 155/295] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:29.691400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5317-dbd2-42ec-9738-634ae975eb62 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:29.692829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5317-dbd2-42ec-9738-634ae975eb62 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:29.692829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5317-dbd2-42ec-9738-634ae975eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:29.710148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c61b5317-dbd2-42ec-9738-634ae975eb62 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:29.712064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c61b5317-dbd2-42ec-9738-634ae975eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:29.782589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 141/296] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:29.808685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1091a8-38f6-4f0c-8373-e5f7133b3bf6 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:29.975912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc138-1eaa-44b0-be7a-79bd40736dde None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:29.976539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc138-1eaa-44b0-be7a-79bd40736dde None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:29.977022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc138-1eaa-44b0-be7a-79bd40736dde None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:29.978630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 142/297] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:29.983881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad14816-2da6-42d2-84b1-cd6af0f4740e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:29.984398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad14816-2da6-42d2-84b1-cd6af0f4740e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:29.984904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad14816-2da6-42d2-84b1-cd6af0f4740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:29.992516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1091a8-38f6-4f0c-8373-e5f7133b3bf6 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:30.003239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-dad14816-2da6-42d2-84b1-cd6af0f4740e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:30.005429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-dad14816-2da6-42d2-84b1-cd6af0f4740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:30.016273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:30.034570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca0a8-ae22-4e0d-a9f4-baf762986fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:30.035373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca0a8-ae22-4e0d-a9f4-baf762986fea None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.035660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca0a8-ae22-4e0d-a9f4-baf762986fea None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.035932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca0a8-ae22-4e0d-a9f4-baf762986fea None admin] PATH_INFO: `/v3/domains/default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.037113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ca0a8-ae22-4e0d-a9f4-baf762986fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:30.038536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ca0a8-ae22-4e0d-a9f4-baf762986fea None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:30.044967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 156/298] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:30.046001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca0a8-ae22-4e0d-a9f4-baf762986fea None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:30.101562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 143/299] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:30.120357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad14816-2da6-42d2-84b1-cd6af0f4740e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:30.168904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:30.263383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad14816-2da6-42d2-84b1-cd6af0f4740e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:30.265518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca0a8-ae22-4e0d-a9f4-baf762986fea None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:30.300679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d145613-b06c-4f8c-9e0b-e0c5ea5d0e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:30.301385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851b708-e3a5-49b0-afae-431b7dff8b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:30.302061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851b708-e3a5-49b0-afae-431b7dff8b0c None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.302318 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d145613-b06c-4f8c-9e0b-e0c5ea5d0e7d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.302417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851b708-e3a5-49b0-afae-431b7dff8b0c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.302612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851b708-e3a5-49b0-afae-431b7dff8b0c None admin] PATH_INFO: `/v3/groups` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.303071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d145613-b06c-4f8c-9e0b-e0c5ea5d0e7d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.303315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851b708-e3a5-49b0-afae-431b7dff8b0c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:30.303730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d145613-b06c-4f8c-9e0b-e0c5ea5d0e7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.304959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851b708-e3a5-49b0-afae-431b7dff8b0c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:30.315907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d145613-b06c-4f8c-9e0b-e0c5ea5d0e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-4d145613-b06c-4f8c-9e0b-e0c5ea5d0e7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:30.324484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:30.325415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:30.326267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:30.327298 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 144/300] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:30.331296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d145613-b06c-4f8c-9e0b-e0c5ea5d0e7d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:30.375346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 157/301] 10.222.0.20 () {60 vars in 1134 bytes} [Thu Jun 16 00:16:30 2022] POST /identity/v3/groups => generated 229 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:30.444873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089bcdf-78c6-483a-9872-b655ed0656a6 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.445160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089bcdf-78c6-483a-9872-b655ed0656a6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.445445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089bcdf-78c6-483a-9872-b655ed0656a6 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.447304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 158/302] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:30.452771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea3bc9-71c4-45ac-b85f-3fcbbec073a3 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.453056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea3bc9-71c4-45ac-b85f-3fcbbec073a3 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.453346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea3bc9-71c4-45ac-b85f-3fcbbec073a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.454745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:30.464020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d145613-b06c-4f8c-9e0b-e0c5ea5d0e7d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:30.481526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-62ea3bc9-71c4-45ac-b85f-3fcbbec073a3 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:30.483017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-62ea3bc9-71c4-45ac-b85f-3fcbbec073a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:30.522105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:30.523086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.523555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.524015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.525442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] RBAC: Adding query filter params (name=reader) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:30.526954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:30.528609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:30.537059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:30.547397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 145/303] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:30.548760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:30.581471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 159/304] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:30.603732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea3bc9-71c4-45ac-b85f-3fcbbec073a3 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:30.689456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1bdf7-d2f6-489d-bb48-a73b1a71e4da None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:30.710897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea3bc9-71c4-45ac-b85f-3fcbbec073a3 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:30.744267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fee56e-e576-41cd-a5b3-db97e718f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:30.749737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fee56e-e576-41cd-a5b3-db97e718f7a2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.750263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fee56e-e576-41cd-a5b3-db97e718f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.750839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fee56e-e576-41cd-a5b3-db97e718f7a2 None admin] PATH_INFO: `/v3/users/041de21f358c4ce59e8f1221d679ad61` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.753850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b153-a39d-4dea-8eda-15ed1f900563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:30.754739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b153-a39d-4dea-8eda-15ed1f900563 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.756222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b153-a39d-4dea-8eda-15ed1f900563 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.756222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b153-a39d-4dea-8eda-15ed1f900563 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.770899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fee56e-e576-41cd-a5b3-db97e718f7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=041de21f358c4ce59e8f1221d679ad61)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:30.771951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826b153-a39d-4dea-8eda-15ed1f900563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:30.772959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fee56e-e576-41cd-a5b3-db97e718f7a2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:30.779603 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 146/305] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:30 2022] GET /identity/v3/users/041de21f358c4ce59e8f1221d679ad61 => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-1826b153-a39d-4dea-8eda-15ed1f900563 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:30.780477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:30.781462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:30.782263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 160/306] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:30.783465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fee56e-e576-41cd-a5b3-db97e718f7a2 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:30.788171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b153-a39d-4dea-8eda-15ed1f900563 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:30.800657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:30.826003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:30.891153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fee56e-e576-41cd-a5b3-db97e718f7a2 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:30.892543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b153-a39d-4dea-8eda-15ed1f900563 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:30.928133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:30.928948 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.929294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.929644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.930471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:30.931383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:30.931494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:30.932362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:30.932843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:30.932959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:30.933426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:30.934679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] RBAC: Adding query filter params (role.id=062e902156a0420f882cc116e074cc88, user.id=041de21f358c4ce59e8f1221d679ad61, scope.system=all) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:30.935168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:30.936886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:30.942333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 161/307] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:30.947213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 147/308] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Jun 16 00:16:30 2022] GET /identity/v3/role_assignments?user.id=041de21f358c4ce59e8f1221d679ad61&scope.system=all&role.id=062e902156a0420f882cc116e074cc88 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:30.948596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:31.068958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19fc9e-3ec7-4649-8192-08209f950ae8 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:31.070693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:31.108744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3d409-3259-40df-9944-7f26ccb0b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:31.108744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3d409-3259-40df-9944-7f26ccb0b59f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:31.109117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3d409-3259-40df-9944-7f26ccb0b59f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:31.109490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3d409-3259-40df-9944-7f26ccb0b59f None admin] PATH_INFO: `/v3/users/57095dd3af49487fa4eccb349a66a3be` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:31.110809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3d409-3259-40df-9944-7f26ccb0b59f None admin] RBAC: Authorizing `identity:get_user(user_id=57095dd3af49487fa4eccb349a66a3be)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:31.112275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3d409-3259-40df-9944-7f26ccb0b59f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:31.113966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 162/309] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:30 2022] GET /identity/v3/users/57095dd3af49487fa4eccb349a66a3be => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:31.120194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:31.216907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:31.225569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49dc5f-bd01-4320-bbe5-e9c54bb670ff None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:31.262777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a228a-f716-40e4-b44c-8604b4e2f9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:31.263695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a228a-f716-40e4-b44c-8604b4e2f9b9 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:31.263989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a228a-f716-40e4-b44c-8604b4e2f9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:31.264271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a228a-f716-40e4-b44c-8604b4e2f9b9 None admin] PATH_INFO: `/v3/system/users/57095dd3af49487fa4eccb349a66a3be/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:31.265828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a228a-f716-40e4-b44c-8604b4e2f9b9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=57095dd3af49487fa4eccb349a66a3be, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:31.267244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a228a-f716-40e4-b44c-8604b4e2f9b9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:31.340456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2b73b-3321-4f35-b2dc-c77f281ce912 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:31.340763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2b73b-3321-4f35-b2dc-c77f281ce912 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:31.340978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2b73b-3321-4f35-b2dc-c77f281ce912 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:31.342484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 163/310] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:31.347030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa99d6-3608-4057-a9bd-b11c5cb82fc2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:31.347324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa99d6-3608-4057-a9bd-b11c5cb82fc2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:31.347604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa99d6-3608-4057-a9bd-b11c5cb82fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:31.352278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 148/311] 10.222.0.20 () {58 vars in 1334 bytes} [Thu Jun 16 00:16:31 2022] PUT /identity/v3/system/users/57095dd3af49487fa4eccb349a66a3be/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:31.367384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e9fa99d6-3608-4057-a9bd-b11c5cb82fc2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:31.369660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e9fa99d6-3608-4057-a9bd-b11c5cb82fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:31.472640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 164/312] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:31.501720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a228a-f716-40e4-b44c-8604b4e2f9b9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:31.609979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a228a-f716-40e4-b44c-8604b4e2f9b9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:31.644222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc8cd4-85a6-45e6-97e2-e07c7e429d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:31.645066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc8cd4-85a6-45e6-97e2-e07c7e429d8f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:31.645355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc8cd4-85a6-45e6-97e2-e07c7e429d8f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:31.645636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc8cd4-85a6-45e6-97e2-e07c7e429d8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:31.656111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc8cd4-85a6-45e6-97e2-e07c7e429d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-c1dc8cd4-85a6-45e6-97e2-e07c7e429d8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:31.663753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:31.664690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:31.665539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 149/313] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:31.671111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa99d6-3608-4057-a9bd-b11c5cb82fc2 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:31.779434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:31.785562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa99d6-3608-4057-a9bd-b11c5cb82fc2 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:31.821272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:31.822507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:31.822790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:31.823068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:31.823818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:31.824125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:31.825696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:31.834311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 165/314] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:31.839569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc8cd4-85a6-45e6-97e2-e07c7e429d8f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:31.843930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:31.944919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc8cd4-85a6-45e6-97e2-e07c7e429d8f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:31.984300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9506b8-f916-481c-8a03-babdacf5511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:31.985630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9506b8-f916-481c-8a03-babdacf5511b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:31.986125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9506b8-f916-481c-8a03-babdacf5511b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:31.986657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9506b8-f916-481c-8a03-babdacf5511b None admin] PATH_INFO: `/v3/users/d2df74a2d11543b9af02b10682ee058b` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:31.988077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9506b8-f916-481c-8a03-babdacf5511b None admin] RBAC: Authorizing `identity:get_user(user_id=d2df74a2d11543b9af02b10682ee058b)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:31.989792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9506b8-f916-481c-8a03-babdacf5511b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:31.991980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 150/315] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:31 2022] GET /identity/v3/users/d2df74a2d11543b9af02b10682ee058b => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:32.000407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:32.103966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30c6a4-e743-499d-91a0-fb8205fcf714 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:32.138816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443af7d-e8af-4c7d-a65d-419bf332157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:32.139488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443af7d-e8af-4c7d-a65d-419bf332157e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:32.139774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443af7d-e8af-4c7d-a65d-419bf332157e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:32.140056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443af7d-e8af-4c7d-a65d-419bf332157e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:32.173221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:32.215762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443af7d-e8af-4c7d-a65d-419bf332157e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:32.217173 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443af7d-e8af-4c7d-a65d-419bf332157e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:32.318376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 166/316] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:32.323912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9506b8-f916-481c-8a03-babdacf5511b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:32.395508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:32.427860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9506b8-f916-481c-8a03-babdacf5511b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:32.461762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009832f9-6275-46d8-b3fa-7e69bd9302f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:32.463154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009832f9-6275-46d8-b3fa-7e69bd9302f9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:32.463441 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009832f9-6275-46d8-b3fa-7e69bd9302f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:32.463720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009832f9-6275-46d8-b3fa-7e69bd9302f9 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:32.465076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009832f9-6275-46d8-b3fa-7e69bd9302f9 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:32.466587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009832f9-6275-46d8-b3fa-7e69bd9302f9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:32.468588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 151/317] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:32 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:32.475133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443af7d-e8af-4c7d-a65d-419bf332157e None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:32.536967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:32.579632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443af7d-e8af-4c7d-a65d-419bf332157e None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:32.616475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e9aa-985e-4145-a350-3bda76284ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:32.617251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e9aa-985e-4145-a350-3bda76284ca1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:32.617673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e9aa-985e-4145-a350-3bda76284ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:32.617954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e9aa-985e-4145-a350-3bda76284ca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:32.618929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e9aa-985e-4145-a350-3bda76284ca1 None admin] RBAC: Adding query filter params (role.id=062e902156a0420f882cc116e074cc88, scope.project.id=f03d23cb840648a38723008cbfdbbde7, user.id=d2df74a2d11543b9af02b10682ee058b) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:32.619249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e9aa-985e-4145-a350-3bda76284ca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:32.620785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e9aa-985e-4145-a350-3bda76284ca1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:32.628180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 167/318] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:32 2022] GET /identity/v3/role_assignments?user.id=d2df74a2d11543b9af02b10682ee058b&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=062e902156a0420f882cc116e074cc88 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:32.678566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18266849-f818-4a91-95fd-7faa30b960d4 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:32.678907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18266849-f818-4a91-95fd-7faa30b960d4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:32.679139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18266849-f818-4a91-95fd-7faa30b960d4 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:32.680734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 152/319] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:32.686855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acca7f-bef4-411e-8dd7-4667c4fdeca9 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:32.687145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acca7f-bef4-411e-8dd7-4667c4fdeca9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:32.687477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acca7f-bef4-411e-8dd7-4667c4fdeca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:32.706003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-27acca7f-bef4-411e-8dd7-4667c4fdeca9 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:32.707241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-27acca7f-bef4-411e-8dd7-4667c4fdeca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:32.785795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 168/320] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:32.809410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18266849-f818-4a91-95fd-7faa30b960d4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:32.919858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18266849-f818-4a91-95fd-7faa30b960d4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:32.954792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949ddaa-0e8c-4047-bc6d-67c94940ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:32.955588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ddaa-0e8c-4047-bc6d-67c94940ec64 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:32.955875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ddaa-0e8c-4047-bc6d-67c94940ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:32.956156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ddaa-0e8c-4047-bc6d-67c94940ec64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:32.957343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949ddaa-0e8c-4047-bc6d-67c94940ec64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:32.958811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949ddaa-0e8c-4047-bc6d-67c94940ec64 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:32.960594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 153/321] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:32.966719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27acca7f-bef4-411e-8dd7-4667c4fdeca9 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:33.023032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:33.070870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27acca7f-bef4-411e-8dd7-4667c4fdeca9 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:33.107087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f35688-f444-405c-8dac-3375d2a2638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:33.107989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f35688-f444-405c-8dac-3375d2a2638c None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.108422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f35688-f444-405c-8dac-3375d2a2638c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.108865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f35688-f444-405c-8dac-3375d2a2638c None admin] PATH_INFO: `/v3/groups` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.109695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f35688-f444-405c-8dac-3375d2a2638c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:33.111305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f35688-f444-405c-8dac-3375d2a2638c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:33.202710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a18b4-d53f-402d-83a2-db1c82795b83 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.203002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a18b4-d53f-402d-83a2-db1c82795b83 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.203286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a18b4-d53f-402d-83a2-db1c82795b83 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.204786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 154/322] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:33.209750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5db8-0b03-4293-85ac-3cdee314dd1b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.210044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5db8-0b03-4293-85ac-3cdee314dd1b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.210331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5db8-0b03-4293-85ac-3cdee314dd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.220564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 169/323] 10.222.0.20 () {60 vars in 1134 bytes} [Thu Jun 16 00:16:32 2022] POST /identity/v3/groups => generated 226 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:16:33.222769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:33.228686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-601c5db8-0b03-4293-85ac-3cdee314dd1b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:33.229897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-601c5db8-0b03-4293-85ac-3cdee314dd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:33.305106 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 155/324] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:33.328191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f35688-f444-405c-8dac-3375d2a2638c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:33.439518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f35688-f444-405c-8dac-3375d2a2638c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:33.452753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:33.474077 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550d75d-eb6a-4919-94b2-e32f212dd76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:33.474884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550d75d-eb6a-4919-94b2-e32f212dd76c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.475172 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550d75d-eb6a-4919-94b2-e32f212dd76c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.475509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550d75d-eb6a-4919-94b2-e32f212dd76c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.487697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550d75d-eb6a-4919-94b2-e32f212dd76c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-3550d75d-eb6a-4919-94b2-e32f212dd76c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:33.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:33.500396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:33.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 170/325] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:33.512403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c5db8-0b03-4293-85ac-3cdee314dd1b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:33.623700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a845876-5641-4625-be31-eb3dd8625b01 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.624082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a845876-5641-4625-be31-eb3dd8625b01 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.624374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a845876-5641-4625-be31-eb3dd8625b01 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.625946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 171/326] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:33.632038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfce386-9853-4e5c-b11c-a84df40e4150 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.632601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfce386-9853-4e5c-b11c-a84df40e4150 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.633473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfce386-9853-4e5c-b11c-a84df40e4150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.636471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c5db8-0b03-4293-85ac-3cdee314dd1b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:33.659719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-0dfce386-9853-4e5c-b11c-a84df40e4150 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:33.660963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0dfce386-9853-4e5c-b11c-a84df40e4150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:33.674592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:33.675794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.676278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.676790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.677746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:33.678267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:33.679842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:33.684016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:33.693270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 156/327] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:33.697571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:33.754523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 172/328] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:33.785487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfce386-9853-4e5c-b11c-a84df40e4150 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:33.824816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8dd3-038d-49ce-8a0a-3ce64b787516 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:33.867896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a9aa5-4140-44dc-b4d7-5b2e3e294e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:33.870072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9aa5-4140-44dc-b4d7-5b2e3e294e31 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.870601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9aa5-4140-44dc-b4d7-5b2e3e294e31 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.871045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9aa5-4140-44dc-b4d7-5b2e3e294e31 None admin] PATH_INFO: `/v3/users/041de21f358c4ce59e8f1221d679ad61` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.872402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a9aa5-4140-44dc-b4d7-5b2e3e294e31 None admin] RBAC: Authorizing `identity:get_user(user_id=041de21f358c4ce59e8f1221d679ad61)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:33.876672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a9aa5-4140-44dc-b4d7-5b2e3e294e31 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:33.879245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 157/329] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:33 2022] GET /identity/v3/users/041de21f358c4ce59e8f1221d679ad61 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:33.885253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a9aa5-4140-44dc-b4d7-5b2e3e294e31 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:33.912273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfce386-9853-4e5c-b11c-a84df40e4150 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:33.913895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:33.964366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3ab0-6ff6-4ce9-906e-1409cb2fd6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:33.965206 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3ab0-6ff6-4ce9-906e-1409cb2fd6e3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:33.965501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3ab0-6ff6-4ce9-906e-1409cb2fd6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:33.965770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3ab0-6ff6-4ce9-906e-1409cb2fd6e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:33.978778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d3ab0-6ff6-4ce9-906e-1409cb2fd6e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-dd1d3ab0-6ff6-4ce9-906e-1409cb2fd6e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:33.987495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:33.988402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:33.989259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 173/330] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:33.991348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:33.994072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3ab0-6ff6-4ce9-906e-1409cb2fd6e3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:34.043192 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a9aa5-4140-44dc-b4d7-5b2e3e294e31 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:34.083951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d989b5-a8e0-4b46-b138-a679edd96193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:34.085479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d989b5-a8e0-4b46-b138-a679edd96193 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:34.085969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d989b5-a8e0-4b46-b138-a679edd96193 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:34.086470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d989b5-a8e0-4b46-b138-a679edd96193 None admin] PATH_INFO: `/v3/system/users/041de21f358c4ce59e8f1221d679ad61/roles/062e902156a0420f882cc116e074cc88` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:34.089090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d989b5-a8e0-4b46-b138-a679edd96193 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=041de21f358c4ce59e8f1221d679ad61, role_id=062e902156a0420f882cc116e074cc88)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:34.091566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d989b5-a8e0-4b46-b138-a679edd96193 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:34.131577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3ab0-6ff6-4ce9-906e-1409cb2fd6e3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:34.152223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 158/331] 10.222.0.20 () {58 vars in 1334 bytes} [Thu Jun 16 00:16:33 2022] PUT /identity/v3/system/users/041de21f358c4ce59e8f1221d679ad61/roles/062e902156a0420f882cc116e074cc88 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:16:34.174608 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:34.175719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:34.176235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:34.176755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:34.177896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:34.178442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:34.180089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:34.191096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 174/332] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:34.196924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d989b5-a8e0-4b46-b138-a679edd96193 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:34.315311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d989b5-a8e0-4b46-b138-a679edd96193 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:34.347590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:34.353777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9ac9-e85f-49c5-bb35-77e337582e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:34.354670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9ac9-e85f-49c5-bb35-77e337582e4c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:34.354957 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9ac9-e85f-49c5-bb35-77e337582e4c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:34.355240 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9ac9-e85f-49c5-bb35-77e337582e4c None admin] PATH_INFO: `/v3/users/57095dd3af49487fa4eccb349a66a3be` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:34.356554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9ac9-e85f-49c5-bb35-77e337582e4c None admin] RBAC: Authorizing `identity:get_user(user_id=57095dd3af49487fa4eccb349a66a3be)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:34.358043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9ac9-e85f-49c5-bb35-77e337582e4c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:34.360140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 159/333] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:34 2022] GET /identity/v3/users/57095dd3af49487fa4eccb349a66a3be => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:34.366096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:34.399243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:34.490682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59eac6-7c87-45f5-9f7a-47ef8b490ea6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:34.531879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a0aa6-57b2-4725-a500-ad514aa873e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:34.532562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a0aa6-57b2-4725-a500-ad514aa873e1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:34.532854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a0aa6-57b2-4725-a500-ad514aa873e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:34.533132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a0aa6-57b2-4725-a500-ad514aa873e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:34.533977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a0aa6-57b2-4725-a500-ad514aa873e1 None admin] RBAC: Adding query filter params (role.id=64517449a51047a3b2ce1079b4912f57, user.id=57095dd3af49487fa4eccb349a66a3be, scope.system=all) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:34.534293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a0aa6-57b2-4725-a500-ad514aa873e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:34.535742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a0aa6-57b2-4725-a500-ad514aa873e1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:34.549602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 175/334] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Jun 16 00:16:34 2022] GET /identity/v3/role_assignments?user.id=57095dd3af49487fa4eccb349a66a3be&scope.system=all&role.id=64517449a51047a3b2ce1079b4912f57 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:36.392313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9b57e-96f6-49e3-a705-1367edd567df None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:36.392842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9b57e-96f6-49e3-a705-1367edd567df None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:36.392917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9b57e-96f6-49e3-a705-1367edd567df None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:36.394534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 160/335] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:36.406085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e28a-dec4-46f3-8a3f-ecac76dd0706 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:36.406588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e28a-dec4-46f3-8a3f-ecac76dd0706 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:36.406666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e28a-dec4-46f3-8a3f-ecac76dd0706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:36.434591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4ae0e28a-dec4-46f3-8a3f-ecac76dd0706 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:36.435792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4ae0e28a-dec4-46f3-8a3f-ecac76dd0706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:36.529235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 176/336] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:36.568821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9b57e-96f6-49e3-a705-1367edd567df None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:36.675044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a538525-70c9-49d9-9465-277f0b7624f7 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:36.675251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a538525-70c9-49d9-9465-277f0b7624f7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:36.675808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a538525-70c9-49d9-9465-277f0b7624f7 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:36.677814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 177/337] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:36.686203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cf412-aa88-4bdc-8de9-ca4d4784f750 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:36.687008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cf412-aa88-4bdc-8de9-ca4d4784f750 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:36.687492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cf412-aa88-4bdc-8de9-ca4d4784f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:36.709787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6e0cf412-aa88-4bdc-8de9-ca4d4784f750 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:36.709997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9b57e-96f6-49e3-a705-1367edd567df None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:36.711704 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6e0cf412-aa88-4bdc-8de9-ca4d4784f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:36.726993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:36.770025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7cd97-0197-4c82-9b56-1ec43f18ccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:36.770825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7cd97-0197-4c82-9b56-1ec43f18ccc1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:36.771105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7cd97-0197-4c82-9b56-1ec43f18ccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:36.771382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7cd97-0197-4c82-9b56-1ec43f18ccc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:36.799929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7cd97-0197-4c82-9b56-1ec43f18ccc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-61a7cd97-0197-4c82-9b56-1ec43f18ccc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:36.809981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:36.810902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:36.811700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 161/338] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:36.821086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7cd97-0197-4c82-9b56-1ec43f18ccc1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:36.848643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 178/339] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:36.899020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cf412-aa88-4bdc-8de9-ca4d4784f750 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:36.930826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:37.007407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7cd97-0197-4c82-9b56-1ec43f18ccc1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:37.014883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cf412-aa88-4bdc-8de9-ca4d4784f750 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:37.058531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65527ccc-9f70-47ec-9380-8291d74659de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:37.059337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65527ccc-9f70-47ec-9380-8291d74659de None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.059619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65527ccc-9f70-47ec-9380-8291d74659de None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.059895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65527ccc-9f70-47ec-9380-8291d74659de None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.076543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:37.077222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.077512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.077787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.078541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:37.078849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:37.080227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:37.090202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65527ccc-9f70-47ec-9380-8291d74659de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-65527ccc-9f70-47ec-9380-8291d74659de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:37.098284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:37.099212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:37.100012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 179/340] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:37.105275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 162/341] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:37.105414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b75a28-6e1a-4af1-8859-32a04e70df5a None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.105414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b75a28-6e1a-4af1-8859-32a04e70df5a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.105414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b75a28-6e1a-4af1-8859-32a04e70df5a None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.106977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 180/342] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:37.111231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:37.111939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc91f20-1b30-4bd2-80fd-1afa4032864f None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.112236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc91f20-1b30-4bd2-80fd-1afa4032864f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.112515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc91f20-1b30-4bd2-80fd-1afa4032864f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.125674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:37.151232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-bbc91f20-1b30-4bd2-80fd-1afa4032864f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:37.153739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-bbc91f20-1b30-4bd2-80fd-1afa4032864f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:37.259772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 181/343] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:37.260969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc91f20-1b30-4bd2-80fd-1afa4032864f None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:37.266648 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070909af-b61a-4fa7-a44d-7301230f5da6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:37.316937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:37.362654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:37.363498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.363606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.363880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.364706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] RBAC: Adding query filter params (name=member) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:37.364965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:37.366333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:37.370340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc91f20-1b30-4bd2-80fd-1afa4032864f None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:37.384428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 163/344] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:37.385407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:37.403366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:37.417267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffba423-e2be-4945-836f-cc57ebfc1fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:37.418081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffba423-e2be-4945-836f-cc57ebfc1fa3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.418467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffba423-e2be-4945-836f-cc57ebfc1fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.418861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffba423-e2be-4945-836f-cc57ebfc1fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.540085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffba423-e2be-4945-836f-cc57ebfc1fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:37.541825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffba423-e2be-4945-836f-cc57ebfc1fa3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:37.593789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f4723-e865-4e26-9b47-59349a3ab499 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:37.662916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 182/345] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:37.664145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffba423-e2be-4945-836f-cc57ebfc1fa3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:37.698193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639d75-82fc-41df-ab84-1994cbde4902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:37.699688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639d75-82fc-41df-ab84-1994cbde4902 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.699688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639d75-82fc-41df-ab84-1994cbde4902 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.699688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639d75-82fc-41df-ab84-1994cbde4902 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.702164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:37.728493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59639d75-82fc-41df-ab84-1994cbde4902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-59639d75-82fc-41df-ab84-1994cbde4902 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:37.737159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:37.738488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:37.739549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:37.741297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 164/346] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:37.742272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639d75-82fc-41df-ab84-1994cbde4902 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:37.792997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:37.803546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffba423-e2be-4945-836f-cc57ebfc1fa3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:37.846234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec40610-e4e9-40b5-a795-9798421a31ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:37.846903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec40610-e4e9-40b5-a795-9798421a31ac None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.847392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec40610-e4e9-40b5-a795-9798421a31ac None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.847582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec40610-e4e9-40b5-a795-9798421a31ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.868884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639d75-82fc-41df-ab84-1994cbde4902 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:37.909420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18771f1d-c55f-4693-8865-ab6e8f099607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:37.910289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771f1d-c55f-4693-8865-ab6e8f099607 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.910579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771f1d-c55f-4693-8865-ab6e8f099607 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.910857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771f1d-c55f-4693-8865-ab6e8f099607 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.912053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18771f1d-c55f-4693-8865-ab6e8f099607 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:37.913520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18771f1d-c55f-4693-8865-ab6e8f099607 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:37.915775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 165/347] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:37 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:37.917156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a18a26-7d03-4aa2-80d1-f35e32562007 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:37.917432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a18a26-7d03-4aa2-80d1-f35e32562007 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:37.917715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a18a26-7d03-4aa2-80d1-f35e32562007 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:37.919095 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 166/348] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:37.919726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a18a26-7d03-4aa2-80d1-f35e32562007 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:37.938136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec40610-e4e9-40b5-a795-9798421a31ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:37.939591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec40610-e4e9-40b5-a795-9798421a31ac None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:37.971680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:38.045145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a18a26-7d03-4aa2-80d1-f35e32562007 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:38.056080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:38.060152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 183/349] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:38.061218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ce90-1fd1-4216-8660-7bc0ccb4061f None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.061515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ce90-1fd1-4216-8660-7bc0ccb4061f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.061797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ce90-1fd1-4216-8660-7bc0ccb4061f None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.063142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 184/350] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:38.071934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ebd9-f1fe-4fee-b525-943571e77497 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.071934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ebd9-f1fe-4fee-b525-943571e77497 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.071934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ebd9-f1fe-4fee-b525-943571e77497 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.071934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 185/351] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:38.071934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ebd9-f1fe-4fee-b525-943571e77497 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:38.089016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834602e8-0eb5-4d45-9f9b-3ef98626878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:38.089706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834602e8-0eb5-4d45-9f9b-3ef98626878a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.090439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834602e8-0eb5-4d45-9f9b-3ef98626878a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.090722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834602e8-0eb5-4d45-9f9b-3ef98626878a None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.091483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834602e8-0eb5-4d45-9f9b-3ef98626878a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:38.091792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834602e8-0eb5-4d45-9f9b-3ef98626878a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:38.093511 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834602e8-0eb5-4d45-9f9b-3ef98626878a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:38.104388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 167/352] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:38.106724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfedafc-74df-4a65-aa82-d8d059878f34 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.106724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfedafc-74df-4a65-aa82-d8d059878f34 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.106724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfedafc-74df-4a65-aa82-d8d059878f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.124895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-dbfedafc-74df-4a65-aa82-d8d059878f34 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:38.126107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-dbfedafc-74df-4a65-aa82-d8d059878f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:38.193037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ebd9-f1fe-4fee-b525-943571e77497 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:38.205851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 168/353] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:38.206717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfedafc-74df-4a65-aa82-d8d059878f34 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:38.227333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:38.231822 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512599d0-833b-42e8-87f8-8185840ecfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:38.233119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512599d0-833b-42e8-87f8-8185840ecfb5 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.233475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512599d0-833b-42e8-87f8-8185840ecfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.237454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512599d0-833b-42e8-87f8-8185840ecfb5 None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.240988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512599d0-833b-42e8-87f8-8185840ecfb5 None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:38.242503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512599d0-833b-42e8-87f8-8185840ecfb5 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:38.244921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 186/354] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:38 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:38.246037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393f915-9854-495d-8845-60fbfd0af662 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.246389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393f915-9854-495d-8845-60fbfd0af662 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.246680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393f915-9854-495d-8845-60fbfd0af662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.265002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e393f915-9854-495d-8845-60fbfd0af662 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:38.266208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e393f915-9854-495d-8845-60fbfd0af662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:38.322587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfedafc-74df-4a65-aa82-d8d059878f34 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:38.324258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:38.347690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 187/355] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:38.349010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a261e3f-79d3-405a-b4db-882dabba832b None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.350194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a261e3f-79d3-405a-b4db-882dabba832b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.350194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a261e3f-79d3-405a-b4db-882dabba832b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.367687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8b9a-7288-4c3f-9294-4ba49bb4cb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:38.369016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-9a261e3f-79d3-405a-b4db-882dabba832b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:38.369123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8b9a-7288-4c3f-9294-4ba49bb4cb81 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.369399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8b9a-7288-4c3f-9294-4ba49bb4cb81 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.369678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8b9a-7288-4c3f-9294-4ba49bb4cb81 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.370213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9a261e3f-79d3-405a-b4db-882dabba832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:38.371031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce8b9a-7288-4c3f-9294-4ba49bb4cb81 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:38.373852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce8b9a-7288-4c3f-9294-4ba49bb4cb81 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:38.375604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 169/356] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:38 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:38.377344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8b9a-7288-4c3f-9294-4ba49bb4cb81 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:38.457253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 188/357] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:38.461374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a261e3f-79d3-405a-b4db-882dabba832b None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:38.495922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8b9a-7288-4c3f-9294-4ba49bb4cb81 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:38.528153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:38.533811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851b9ce-c789-4118-8e97-0b23409e060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:38.534469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851b9ce-c789-4118-8e97-0b23409e060d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.534750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851b9ce-c789-4118-8e97-0b23409e060d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.535031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851b9ce-c789-4118-8e97-0b23409e060d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.581444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a261e3f-79d3-405a-b4db-882dabba832b None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:38.585771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:38.619699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851b9ce-c789-4118-8e97-0b23409e060d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:38.621466 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851b9ce-c789-4118-8e97-0b23409e060d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:38.623742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ddf1d-b0a4-4461-af44-09b4fe862010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:38.624584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddf1d-b0a4-4461-af44-09b4fe862010 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.624875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddf1d-b0a4-4461-af44-09b4fe862010 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.625158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddf1d-b0a4-4461-af44-09b4fe862010 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.637877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ddf1d-b0a4-4461-af44-09b4fe862010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-a70ddf1d-b0a4-4461-af44-09b4fe862010 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:38.646182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:38.647221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:38.648198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 189/358] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:38.650571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ddf1d-b0a4-4461-af44-09b4fe862010 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:38.745891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 170/359] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:38.747107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851b9ce-c789-4118-8e97-0b23409e060d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:38.772017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ddf1d-b0a4-4461-af44-09b4fe862010 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:38.789795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:38.812115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:38.812999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.813278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.813551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.814386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] RBAC: Adding query filter params (group.id=fee321acb6694877845fc83e036f9faf, role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=bcf793428063417f9cb53b3d0b4eea85) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:38.814697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:38.816143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:38.826042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 190/360] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:38 2022] GET /identity/v3/role_assignments?group.id=fee321acb6694877845fc83e036f9faf&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=64517449a51047a3b2ce1079b4912f57 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:38.826868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:38.833020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:38.888097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851b9ce-c789-4118-8e97-0b23409e060d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:38.942733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0559481-a594-436a-a96b-e8f5f1a2edba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:38.943943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0559481-a594-436a-a96b-e8f5f1a2edba None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:38.944229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0559481-a594-436a-a96b-e8f5f1a2edba None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:38.944896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0559481-a594-436a-a96b-e8f5f1a2edba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:38.957619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0559481-a594-436a-a96b-e8f5f1a2edba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d0559481-a594-436a-a96b-e8f5f1a2edba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:38.966380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:38.967362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:16:38.968187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:38.968859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 171/361] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:38.970341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0559481-a594-436a-a96b-e8f5f1a2edba None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:38.978758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0987f-ba8d-4b95-9283-5d8b234dccb9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.024005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe93a5-ef25-4bfd-afe2-2c4d987c1558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.025583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe93a5-ef25-4bfd-afe2-2c4d987c1558 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.025875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe93a5-ef25-4bfd-afe2-2c4d987c1558 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.026159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe93a5-ef25-4bfd-afe2-2c4d987c1558 None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.027354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe93a5-ef25-4bfd-afe2-2c4d987c1558 None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.029555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe93a5-ef25-4bfd-afe2-2c4d987c1558 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.030638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 191/362] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:38 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:39.031252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe93a5-ef25-4bfd-afe2-2c4d987c1558 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.096267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:39.100023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0559481-a594-436a-a96b-e8f5f1a2edba None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.136471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:39.145269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52dad-8596-4560-8d34-3bad684a8d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.146808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52dad-8596-4560-8d34-3bad684a8d50 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.147415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52dad-8596-4560-8d34-3bad684a8d50 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.147903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52dad-8596-4560-8d34-3bad684a8d50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.160583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe93a5-ef25-4bfd-afe2-2c4d987c1558 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.168467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e52dad-8596-4560-8d34-3bad684a8d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-94e52dad-8596-4560-8d34-3bad684a8d50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:39.176833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:39.177822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:39.178627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:39.180006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 172/363] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:39.180118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52dad-8596-4560-8d34-3bad684a8d50 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.204088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.205140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.205613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.206051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.206974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:39.207459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.209045 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.223488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 192/364] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:39.225556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.320468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52dad-8596-4560-8d34-3bad684a8d50 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.334342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541a647-0865-4a50-923d-090c2aa7f2a3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.346485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:39.358931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d6bfd-e524-451a-b2c1-36f1448b268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.360109 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d6bfd-e524-451a-b2c1-36f1448b268a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.360587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d6bfd-e524-451a-b2c1-36f1448b268a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.361007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d6bfd-e524-451a-b2c1-36f1448b268a None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.362329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d6bfd-e524-451a-b2c1-36f1448b268a None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.363891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d6bfd-e524-451a-b2c1-36f1448b268a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.365830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 173/365] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:39.371161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.371594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d6bfd-e524-451a-b2c1-36f1448b268a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.371891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.372331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.372431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.373226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:39.373531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.374902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.385359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 193/366] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:39.386170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.438848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:39.481185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d6bfd-e524-451a-b2c1-36f1448b268a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.490012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e90b65-5ae2-48d8-a7cf-89346c88e464 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.524493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.525515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.525978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.526694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.527656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] RBAC: Adding query filter params (group.id=fee321acb6694877845fc83e036f9faf, role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=f03d23cb840648a38723008cbfdbbde7) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:39.528208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.530633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.544575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 174/367] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/role_assignments?group.id=fee321acb6694877845fc83e036f9faf&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=64517449a51047a3b2ce1079b4912f57 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:39.544575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.549719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.552547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.552547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.552547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.552547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:39.552547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.553444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.568733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 194/368] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:39.569784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.624631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:39.686494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fbfad-904f-473e-ae2c-45be140affe2 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.715323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218aa5-b574-4afc-a0a8-70891c1af9a4 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.720409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:39.739468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0e7c0-b95c-420d-8797-13ae6d7f9914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.745741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0e7c0-b95c-420d-8797-13ae6d7f9914 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.745741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0e7c0-b95c-420d-8797-13ae6d7f9914 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.745741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0e7c0-b95c-420d-8797-13ae6d7f9914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.764506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5e615-182c-4fb2-a12d-34843cc5d0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.765864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5e615-182c-4fb2-a12d-34843cc5d0ee None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.766151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5e615-182c-4fb2-a12d-34843cc5d0ee None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.766430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5e615-182c-4fb2-a12d-34843cc5d0ee None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.767882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5e615-182c-4fb2-a12d-34843cc5d0ee None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.770002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5e615-182c-4fb2-a12d-34843cc5d0ee None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.771828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 195/369] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:39.772386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5e615-182c-4fb2-a12d-34843cc5d0ee None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.833578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0e7c0-b95c-420d-8797-13ae6d7f9914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.835923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0e7c0-b95c-420d-8797-13ae6d7f9914 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.898769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5e615-182c-4fb2-a12d-34843cc5d0ee None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:39.900985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:39.940368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141e7dd-e60f-4592-bade-9a2f8057a784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:39.942148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141e7dd-e60f-4592-bade-9a2f8057a784 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:39.942633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141e7dd-e60f-4592-bade-9a2f8057a784 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:39.943545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141e7dd-e60f-4592-bade-9a2f8057a784 None admin] PATH_INFO: `/v3/users/041de21f358c4ce59e8f1221d679ad61` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:39.945256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141e7dd-e60f-4592-bade-9a2f8057a784 None admin] RBAC: Authorizing `identity:get_user(user_id=041de21f358c4ce59e8f1221d679ad61)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:39.947871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141e7dd-e60f-4592-bade-9a2f8057a784 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:39.950669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 196/370] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/users/041de21f358c4ce59e8f1221d679ad61 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:39.951631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141e7dd-e60f-4592-bade-9a2f8057a784 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.967743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 175/371] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:39.968594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0e7c0-b95c-420d-8797-13ae6d7f9914 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:39.995253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:40.076392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141e7dd-e60f-4592-bade-9a2f8057a784 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:40.093406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0e7c0-b95c-420d-8797-13ae6d7f9914 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:40.114328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59bd1-3a10-4ff5-a90b-015e241a6455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:40.114988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59bd1-3a10-4ff5-a90b-015e241a6455 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:40.115278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59bd1-3a10-4ff5-a90b-015e241a6455 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:40.115567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59bd1-3a10-4ff5-a90b-015e241a6455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:40.137487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:40.139003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:40.139647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:40.140234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:40.141546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] RBAC: Adding query filter params (group.id=fee321acb6694877845fc83e036f9faf, role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=bcf793428063417f9cb53b3d0b4eea85) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:40.142458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:40.144617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:40.154218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 176/372] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/role_assignments?group.id=fee321acb6694877845fc83e036f9faf&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:40.155289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:40.158400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:40.233863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab59bd1-3a10-4ff5-a90b-015e241a6455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:40.235715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab59bd1-3a10-4ff5-a90b-015e241a6455 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:40.316560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:40.343284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04f0ce-48b8-4726-9ed0-f312fc137f06 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:40.399524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:40.400184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:40.400468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:40.400775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:40.401624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] RBAC: Adding query filter params (role.id=062e902156a0420f882cc116e074cc88, user.id=041de21f358c4ce59e8f1221d679ad61, scope.system=all) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:40.401939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:40.403863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:40.417117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 197/373] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:40.418212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59bd1-3a10-4ff5-a90b-015e241a6455 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:40.424248 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 177/374] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Jun 16 00:16:40 2022] GET /identity/v3/role_assignments?user.id=041de21f358c4ce59e8f1221d679ad61&scope.system=all&role.id=062e902156a0420f882cc116e074cc88 => generated 510 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:40.425273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:40.556280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:40.617037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7c5d-eb3b-4fb6-bb54-0efad89e38a1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:40.626352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59bd1-3a10-4ff5-a90b-015e241a6455 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:40.682914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee435e3f-68ba-4219-9898-527a1333f4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:40.684975 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bc889-31e0-4520-90d0-b79848d93471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:40.685856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bc889-31e0-4520-90d0-b79848d93471 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:40.686141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bc889-31e0-4520-90d0-b79848d93471 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:40.686420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bc889-31e0-4520-90d0-b79848d93471 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:40.687424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee435e3f-68ba-4219-9898-527a1333f4ca None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:40.687887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee435e3f-68ba-4219-9898-527a1333f4ca None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:40.687965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee435e3f-68ba-4219-9898-527a1333f4ca None admin] PATH_INFO: `/v3/projects/26e616646e114f6f885b0d1df925801d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:40.689354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee435e3f-68ba-4219-9898-527a1333f4ca None admin] RBAC: Authorizing `identity:get_project(project_id=26e616646e114f6f885b0d1df925801d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:40.690824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee435e3f-68ba-4219-9898-527a1333f4ca None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:40.692837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bc889-31e0-4520-90d0-b79848d93471 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:40.694315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bc889-31e0-4520-90d0-b79848d93471 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:40.697432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 178/375] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:40 2022] GET /identity/v3/projects/26e616646e114f6f885b0d1df925801d => generated 344 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:40.697676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 198/376] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:40 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:40.701784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee435e3f-68ba-4219-9898-527a1333f4ca None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:40.707924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bc889-31e0-4520-90d0-b79848d93471 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:40.734740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:40.843900 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee435e3f-68ba-4219-9898-527a1333f4ca None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:40.889721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b2775-abc4-4ca5-aacb-64d8fc463928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:40.891490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2775-abc4-4ca5-aacb-64d8fc463928 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:40.892226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2775-abc4-4ca5-aacb-64d8fc463928 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:40.892874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2775-abc4-4ca5-aacb-64d8fc463928 None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:40.894314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b2775-abc4-4ca5-aacb-64d8fc463928 None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:40.895930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b2775-abc4-4ca5-aacb-64d8fc463928 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:40.898477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 179/377] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:40 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:40.906694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b2775-abc4-4ca5-aacb-64d8fc463928 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:40.941113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bc889-31e0-4520-90d0-b79848d93471 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:40.985578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc09fc3-fee9-478c-b648-031c648116fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:40.989001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc09fc3-fee9-478c-b648-031c648116fe None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:40.989494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc09fc3-fee9-478c-b648-031c648116fe None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:40.991002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc09fc3-fee9-478c-b648-031c648116fe None admin] PATH_INFO: `/v3/groups/8b2ffc8b2b08475dbc227a185caa690b` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:41.003348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc09fc3-fee9-478c-b648-031c648116fe None admin] RBAC: Authorizing `identity:get_group(group_id=8b2ffc8b2b08475dbc227a185caa690b)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:41.005104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc09fc3-fee9-478c-b648-031c648116fe None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:41.010848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 199/378] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:40 2022] GET /identity/v3/groups/8b2ffc8b2b08475dbc227a185caa690b => generated 229 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:41.017272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc09fc3-fee9-478c-b648-031c648116fe None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:41.021196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:41.071126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:41.101109 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b2775-abc4-4ca5-aacb-64d8fc463928 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:41.132680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc09fc3-fee9-478c-b648-031c648116fe None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:41.150756 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000509-502d-4edb-9f91-ae662715850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:41.151425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000509-502d-4edb-9f91-ae662715850f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:41.151710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000509-502d-4edb-9f91-ae662715850f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:41.152089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000509-502d-4edb-9f91-ae662715850f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:41.156906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000509-502d-4edb-9f91-ae662715850f None admin] RBAC: Adding query filter params (group.id=fee321acb6694877845fc83e036f9faf, role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=f03d23cb840648a38723008cbfdbbde7) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:41.157234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000509-502d-4edb-9f91-ae662715850f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:41.158672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000509-502d-4edb-9f91-ae662715850f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:41.164685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 180/379] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:40 2022] GET /identity/v3/role_assignments?group.id=fee321acb6694877845fc83e036f9faf&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:41.179496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a68f6-cc96-4886-b654-9355cd16700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:41.181274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a68f6-cc96-4886-b654-9355cd16700f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:41.181766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a68f6-cc96-4886-b654-9355cd16700f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:41.182316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a68f6-cc96-4886-b654-9355cd16700f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:41.183402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a68f6-cc96-4886-b654-9355cd16700f None admin] RBAC: Adding query filter params (group.id=8b2ffc8b2b08475dbc227a185caa690b, role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=26e616646e114f6f885b0d1df925801d) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:41.183954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a68f6-cc96-4886-b654-9355cd16700f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:41.185694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a68f6-cc96-4886-b654-9355cd16700f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:41.198066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 200/380] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:41 2022] GET /identity/v3/role_assignments?group.id=8b2ffc8b2b08475dbc227a185caa690b&scope.project.id=26e616646e114f6f885b0d1df925801d&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:41.272803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc30fa3-c444-467a-b2d8-e6a2c7f78302 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:41.273137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc30fa3-c444-467a-b2d8-e6a2c7f78302 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:41.273375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc30fa3-c444-467a-b2d8-e6a2c7f78302 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:41.275635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 181/381] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:41.281553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9645af-2050-41aa-b141-4554d6158d8c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:41.281848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9645af-2050-41aa-b141-4554d6158d8c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:41.282138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9645af-2050-41aa-b141-4554d6158d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:41.309856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-be9645af-2050-41aa-b141-4554d6158d8c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:41.311099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-be9645af-2050-41aa-b141-4554d6158d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:41.389973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:41.398651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 201/382] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:41.441375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc30fa3-c444-467a-b2d8-e6a2c7f78302 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:41.584935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc30fa3-c444-467a-b2d8-e6a2c7f78302 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:41.628768 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8be832-ff2b-487c-a57d-5fb463750fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:41.629567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be832-ff2b-487c-a57d-5fb463750fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:41.629850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be832-ff2b-487c-a57d-5fb463750fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:41.630127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be832-ff2b-487c-a57d-5fb463750fb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:41.641457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8be832-ff2b-487c-a57d-5fb463750fb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-ac8be832-ff2b-487c-a57d-5fb463750fb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:41.649867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:41.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:41.652269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 182/383] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:41.657348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9645af-2050-41aa-b141-4554d6158d8c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:41.811987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9645af-2050-41aa-b141-4554d6158d8c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:41.858743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:41.859347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:41.859637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:41.859918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:41.860709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:41.861017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:41.862387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:41.874548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 202/384] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:41.880136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8be832-ff2b-487c-a57d-5fb463750fb9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:41.884536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:42.017004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8be832-ff2b-487c-a57d-5fb463750fb9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:42.064291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79baad7a-66e5-421a-9b8c-930ae270b868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:42.065186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79baad7a-66e5-421a-9b8c-930ae270b868 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.065477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79baad7a-66e5-421a-9b8c-930ae270b868 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.065748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79baad7a-66e5-421a-9b8c-930ae270b868 None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.066958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79baad7a-66e5-421a-9b8c-930ae270b868 None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:42.068411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79baad7a-66e5-421a-9b8c-930ae270b868 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:42.070387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 183/385] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:41 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:42.075536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:42.159051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a4c4-f350-47a7-849a-93c2b3d94043 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.159607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a4c4-f350-47a7-849a-93c2b3d94043 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.160067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a4c4-f350-47a7-849a-93c2b3d94043 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.161862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 184/386] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:42.167806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b4610-818c-425d-8a7e-8781bdf15a9b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.168358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b4610-818c-425d-8a7e-8781bdf15a9b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.168885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b4610-818c-425d-8a7e-8781bdf15a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.181732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:42.187911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-508b4610-818c-425d-8a7e-8781bdf15a9b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:42.194312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-508b4610-818c-425d-8a7e-8781bdf15a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:42.250179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8e56c-8a8d-4c79-b5ab-efc3235cf93b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:42.277219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 185/387] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:42.305047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b4610-818c-425d-8a7e-8781bdf15a9b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:42.307740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:42.316324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ae1f7-ebbb-4f8c-9e7e-a44dc3412094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:42.317242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ae1f7-ebbb-4f8c-9e7e-a44dc3412094 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.320136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ae1f7-ebbb-4f8c-9e7e-a44dc3412094 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.320136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ae1f7-ebbb-4f8c-9e7e-a44dc3412094 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.320136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ae1f7-ebbb-4f8c-9e7e-a44dc3412094 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:42.327486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ae1f7-ebbb-4f8c-9e7e-a44dc3412094 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:42.327486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 203/388] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:42 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:42.328869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ae1f7-ebbb-4f8c-9e7e-a44dc3412094 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:42.422356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b4610-818c-425d-8a7e-8781bdf15a9b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:42.459377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b82fa-159c-4e4d-9e81-3cf375c51fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:42.460613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b82fa-159c-4e4d-9e81-3cf375c51fa1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.461079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b82fa-159c-4e4d-9e81-3cf375c51fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.461562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b82fa-159c-4e4d-9e81-3cf375c51fa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.472649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b82fa-159c-4e4d-9e81-3cf375c51fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d34b82fa-159c-4e4d-9e81-3cf375c51fa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:42.481327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:42.482291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:42.483110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:42.485480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 186/389] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:42.491355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b82fa-159c-4e4d-9e81-3cf375c51fa1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:42.562926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:42.579135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ae1f7-ebbb-4f8c-9e7e-a44dc3412094 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:42.621120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b82fa-159c-4e4d-9e81-3cf375c51fa1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:42.651004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:42.660086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:42.661149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.668975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.668975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.668975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] RBAC: Adding query filter params (name=member) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:42.668975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:42.668975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:42.675385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 187/390] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:42.679002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c636-481c-4878-bb3d-f62afb15a6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:42.680511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c636-481c-4878-bb3d-f62afb15a6b2 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.682038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:42.685779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c636-481c-4878-bb3d-f62afb15a6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.686220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c636-481c-4878-bb3d-f62afb15a6b2 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85/groups/fee321acb6694877845fc83e036f9faf/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.690212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c636-481c-4878-bb3d-f62afb15a6b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcf793428063417f9cb53b3d0b4eea85, group_id=fee321acb6694877845fc83e036f9faf, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:42.691809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c636-481c-4878-bb3d-f62afb15a6b2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:42.767228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 204/391] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jun 16 00:16:42 2022] PUT /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85/groups/fee321acb6694877845fc83e036f9faf/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:16:42.813506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b022210-7876-4ead-ae2f-600581c7fb95 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:42.840665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:42.853073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8a309-ae80-410f-aacb-df0f9ded5da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:42.854171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a309-ae80-410f-aacb-df0f9ded5da7 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.854707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a309-ae80-410f-aacb-df0f9ded5da7 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.855133 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a309-ae80-410f-aacb-df0f9ded5da7 None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.856498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8a309-ae80-410f-aacb-df0f9ded5da7 None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:42.858129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8a309-ae80-410f-aacb-df0f9ded5da7 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:42.860010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 188/392] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:42 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:42.866006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c636-481c-4878-bb3d-f62afb15a6b2 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:42.980716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578b89-134e-4af0-81f2-b37a3f42b3d9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.981570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578b89-134e-4af0-81f2-b37a3f42b3d9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.981995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578b89-134e-4af0-81f2-b37a3f42b3d9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:42.983541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 189/393] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:42.989733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5c4f3-1bdc-441c-9acc-18397636898d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:42.990168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5c4f3-1bdc-441c-9acc-18397636898d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:42.990559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5c4f3-1bdc-441c-9acc-18397636898d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.011854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ebc5c4f3-1bdc-441c-9acc-18397636898d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:43.013639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ebc5c4f3-1bdc-441c-9acc-18397636898d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:43.022008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c636-481c-4878-bb3d-f62afb15a6b2 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:43.097683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc3724-ae77-4d9b-8499-a2505fabe71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:43.098864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc3724-ae77-4d9b-8499-a2505fabe71d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:43.099330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc3724-ae77-4d9b-8499-a2505fabe71d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:43.099783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc3724-ae77-4d9b-8499-a2505fabe71d None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.101190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc3724-ae77-4d9b-8499-a2505fabe71d None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:43.103350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc3724-ae77-4d9b-8499-a2505fabe71d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:43.105560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 205/394] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:42 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:43.115140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc3724-ae77-4d9b-8499-a2505fabe71d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:43.129623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 190/395] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:43.154400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5c4f3-1bdc-441c-9acc-18397636898d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:43.179072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:43.224752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc3724-ae77-4d9b-8499-a2505fabe71d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:43.261860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5c4f3-1bdc-441c-9acc-18397636898d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:43.264256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd4459-0fb4-4af3-b5a7-50731ad8fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:43.267101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd4459-0fb4-4af3-b5a7-50731ad8fb6a None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:43.267101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd4459-0fb4-4af3-b5a7-50731ad8fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:43.267101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd4459-0fb4-4af3-b5a7-50731ad8fb6a None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7/groups/fee321acb6694877845fc83e036f9faf/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.268410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd4459-0fb4-4af3-b5a7-50731ad8fb6a None admin] RBAC: Authorizing `identity:create_grant(project_id=f03d23cb840648a38723008cbfdbbde7, group_id=fee321acb6694877845fc83e036f9faf, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:43.270077 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd4459-0fb4-4af3-b5a7-50731ad8fb6a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:43.286239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:43.301724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab64f33-37f4-452b-8792-0b02a0c2471a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:43.302548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab64f33-37f4-452b-8792-0b02a0c2471a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:43.302836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab64f33-37f4-452b-8792-0b02a0c2471a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:43.303112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab64f33-37f4-452b-8792-0b02a0c2471a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.314381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab64f33-37f4-452b-8792-0b02a0c2471a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-fab64f33-37f4-452b-8792-0b02a0c2471a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:43.323073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:43.324977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:43.325804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 191/396] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:43.334741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab64f33-37f4-452b-8792-0b02a0c2471a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:43.371510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 206/397] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jun 16 00:16:43 2022] PUT /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7/groups/fee321acb6694877845fc83e036f9faf/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:43.476883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab64f33-37f4-452b-8792-0b02a0c2471a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:43.530564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:43.531146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:43.531426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:43.531704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.532480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:43.532817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:43.534227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:43.549505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 192/398] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:43.554702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd4459-0fb4-4af3-b5a7-50731ad8fb6a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:43.700839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd4459-0fb4-4af3-b5a7-50731ad8fb6a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:43.716548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:43.737837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a40f74-603f-4cfd-8eef-931901d37e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:43.738788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a40f74-603f-4cfd-8eef-931901d37e13 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:43.739078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a40f74-603f-4cfd-8eef-931901d37e13 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:43.739358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a40f74-603f-4cfd-8eef-931901d37e13 None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.740582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a40f74-603f-4cfd-8eef-931901d37e13 None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:43.742095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a40f74-603f-4cfd-8eef-931901d37e13 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:43.744604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 207/399] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:43 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:43.750520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:43.808989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:43.854870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d78e7-8cf4-4009-8991-2cf307e2e26c None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:43.855538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d78e7-8cf4-4009-8991-2cf307e2e26c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:43.855832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d78e7-8cf4-4009-8991-2cf307e2e26c None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.857494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 208/400] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:43.873748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a062f-62e7-4cf0-8297-cccc58f8b871 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:43.874048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a062f-62e7-4cf0-8297-cccc58f8b871 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:43.874330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a062f-62e7-4cf0-8297-cccc58f8b871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.898998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c33a062f-62e7-4cf0-8297-cccc58f8b871 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:43.900235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c33a062f-62e7-4cf0-8297-cccc58f8b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:43.904289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04057839-97e7-4bc8-b370-04b6194423e2 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:43.965667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b028184-c779-45c3-9897-a9adb57550b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:43.966538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b028184-c779-45c3-9897-a9adb57550b8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:43.966829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b028184-c779-45c3-9897-a9adb57550b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:43.967103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b028184-c779-45c3-9897-a9adb57550b8 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:43.968863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b028184-c779-45c3-9897-a9adb57550b8 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:43.970302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b028184-c779-45c3-9897-a9adb57550b8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:43.972158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 193/401] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:43 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:43.978619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b028184-c779-45c3-9897-a9adb57550b8 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:44.013882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 209/402] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:44.049652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a062f-62e7-4cf0-8297-cccc58f8b871 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:44.140238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b028184-c779-45c3-9897-a9adb57550b8 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:44.157257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:44.178053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a062f-62e7-4cf0-8297-cccc58f8b871 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:44.184028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1866e-399e-47ad-954a-4ec4ebdc06c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:44.185073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1866e-399e-47ad-954a-4ec4ebdc06c5 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:44.185453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1866e-399e-47ad-954a-4ec4ebdc06c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:44.185829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1866e-399e-47ad-954a-4ec4ebdc06c5 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85/groups/fee321acb6694877845fc83e036f9faf/roles/96431666cb824a7087dbd9f0e30c0c80` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:44.188749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1866e-399e-47ad-954a-4ec4ebdc06c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcf793428063417f9cb53b3d0b4eea85, group_id=fee321acb6694877845fc83e036f9faf, role_id=96431666cb824a7087dbd9f0e30c0c80)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:44.191127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1866e-399e-47ad-954a-4ec4ebdc06c5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:44.193707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:44.216282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791a621-5b26-4a90-b4ab-810274eb570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:44.217202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a621-5b26-4a90-b4ab-810274eb570e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:44.217583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a621-5b26-4a90-b4ab-810274eb570e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:44.217936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a621-5b26-4a90-b4ab-810274eb570e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:44.228476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791a621-5b26-4a90-b4ab-810274eb570e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-4791a621-5b26-4a90-b4ab-810274eb570e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:44.236759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:44.237707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:44.238872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:44.243568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 210/403] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:44.244895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9aa1a-e6a3-4be5-84b1-4e6f42a88411 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:44.245190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9aa1a-e6a3-4be5-84b1-4e6f42a88411 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:44.245478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9aa1a-e6a3-4be5-84b1-4e6f42a88411 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:44.250646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 211/404] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:44.250646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9aa1a-e6a3-4be5-84b1-4e6f42a88411 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:44.289454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 194/405] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jun 16 00:16:43 2022] PUT /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85/groups/fee321acb6694877845fc83e036f9faf/roles/96431666cb824a7087dbd9f0e30c0c80 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:16:44.291921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab3446-fa6c-4483-ba94-f7fea193ed39 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:44.292429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab3446-fa6c-4483-ba94-f7fea193ed39 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:44.293586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab3446-fa6c-4483-ba94-f7fea193ed39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:44.320642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-94ab3446-fa6c-4483-ba94-f7fea193ed39 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:44.324072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-94ab3446-fa6c-4483-ba94-f7fea193ed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:44.423243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9aa1a-e6a3-4be5-84b1-4e6f42a88411 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:44.481107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 195/406] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:44.499620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:44.500302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:44.500623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:44.500904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:44.501656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:44.501968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:44.503343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:44.510401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab3446-fa6c-4483-ba94-f7fea193ed39 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:44.516636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 212/407] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:44.518209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:44.554584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:44.623405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab3446-fa6c-4483-ba94-f7fea193ed39 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:44.626171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:44.630787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164427e-f0f6-4f3b-b8bf-b67547dfb44d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:44.668254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b87c4d-71e6-4721-93cf-357e5e22b53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:44.669590 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b87c4d-71e6-4721-93cf-357e5e22b53d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:44.670057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b87c4d-71e6-4721-93cf-357e5e22b53d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:44.670480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b87c4d-71e6-4721-93cf-357e5e22b53d None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:44.671818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b87c4d-71e6-4721-93cf-357e5e22b53d None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:44.673752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b87c4d-71e6-4721-93cf-357e5e22b53d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:44.680110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9383e91-4134-40d8-aaed-bc2cf7e41021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:44.680110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383e91-4134-40d8-aaed-bc2cf7e41021 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:44.680110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383e91-4134-40d8-aaed-bc2cf7e41021 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:44.680110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383e91-4134-40d8-aaed-bc2cf7e41021 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:44.682228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 213/408] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:44 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:44.684058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b87c4d-71e6-4721-93cf-357e5e22b53d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:44.702545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9383e91-4134-40d8-aaed-bc2cf7e41021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-f9383e91-4134-40d8-aaed-bc2cf7e41021 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:44.716235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:44.717370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:44.718669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 196/409] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:44.723996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9383e91-4134-40d8-aaed-bc2cf7e41021 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:44.793969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b87c4d-71e6-4721-93cf-357e5e22b53d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:44.808851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:44.856428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164257a-de05-4c3d-b42d-82151ee5afa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:44.857320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164257a-de05-4c3d-b42d-82151ee5afa7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:44.857605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164257a-de05-4c3d-b42d-82151ee5afa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:44.857883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164257a-de05-4c3d-b42d-82151ee5afa7 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:44.859143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164257a-de05-4c3d-b42d-82151ee5afa7 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:44.860601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164257a-de05-4c3d-b42d-82151ee5afa7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:44.862294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 214/410] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:44 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:44.870536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164257a-de05-4c3d-b42d-82151ee5afa7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:44.920171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9383e91-4134-40d8-aaed-bc2cf7e41021 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:45.003914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164257a-de05-4c3d-b42d-82151ee5afa7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:45.015972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:45.016684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:45.016976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:45.017265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:45.018029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:45.018342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:45.019733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:45.036447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 197/411] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:45.042151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:45.042574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc5cd4-730a-4b97-bbb1-c45b2b797508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:45.043472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc5cd4-730a-4b97-bbb1-c45b2b797508 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:45.043757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc5cd4-730a-4b97-bbb1-c45b2b797508 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:45.044031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc5cd4-730a-4b97-bbb1-c45b2b797508 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7/groups/fee321acb6694877845fc83e036f9faf/roles/96431666cb824a7087dbd9f0e30c0c80` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:45.046331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc5cd4-730a-4b97-bbb1-c45b2b797508 None admin] RBAC: Authorizing `identity:create_grant(project_id=f03d23cb840648a38723008cbfdbbde7, group_id=fee321acb6694877845fc83e036f9faf, role_id=96431666cb824a7087dbd9f0e30c0c80)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:45.048069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc5cd4-730a-4b97-bbb1-c45b2b797508 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:45.079036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:45.120668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 215/412] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jun 16 00:16:44 2022] PUT /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7/groups/fee321acb6694877845fc83e036f9faf/roles/96431666cb824a7087dbd9f0e30c0c80 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:45.153291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f56e-9d2a-4c47-bb09-cb5715441232 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:45.190803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7459f-4705-4c04-92fc-f3d37fd67f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:45.191708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7459f-4705-4c04-92fc-f3d37fd67f26 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:45.191976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7459f-4705-4c04-92fc-f3d37fd67f26 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:45.192257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7459f-4705-4c04-92fc-f3d37fd67f26 None admin] PATH_INFO: `/v3/groups/8b2ffc8b2b08475dbc227a185caa690b` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:45.193494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7459f-4705-4c04-92fc-f3d37fd67f26 None admin] RBAC: Authorizing `identity:get_group(group_id=8b2ffc8b2b08475dbc227a185caa690b)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:45.194944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7459f-4705-4c04-92fc-f3d37fd67f26 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:45.196928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 198/413] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:45 2022] GET /identity/v3/groups/8b2ffc8b2b08475dbc227a185caa690b => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:45.205864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc5cd4-730a-4b97-bbb1-c45b2b797508 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:45.261751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:45.359560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc5cd4-730a-4b97-bbb1-c45b2b797508 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:45.397106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b37b8-d54c-44df-98b6-a32b76c98965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:45.397977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b37b8-d54c-44df-98b6-a32b76c98965 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:45.398326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b37b8-d54c-44df-98b6-a32b76c98965 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:45.398601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b37b8-d54c-44df-98b6-a32b76c98965 None admin] PATH_INFO: `/v3/projects/26e616646e114f6f885b0d1df925801d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:45.400267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b37b8-d54c-44df-98b6-a32b76c98965 None admin] RBAC: Authorizing `identity:get_project(project_id=26e616646e114f6f885b0d1df925801d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:45.403132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b37b8-d54c-44df-98b6-a32b76c98965 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:45.406025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 216/414] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:45 2022] GET /identity/v3/projects/26e616646e114f6f885b0d1df925801d => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:45.412104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7459f-4705-4c04-92fc-f3d37fd67f26 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:45.551063 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:45.562532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7459f-4705-4c04-92fc-f3d37fd67f26 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:45.585924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c94f1-8bad-407f-a6db-e0e7d322ac7f None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:45.586244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c94f1-8bad-407f-a6db-e0e7d322ac7f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:45.586494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c94f1-8bad-407f-a6db-e0e7d322ac7f None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:45.587964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 217/415] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:16:45.593273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3531ef-4ecd-44ff-bcc7-d3d49af3eb86 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:45.593569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3531ef-4ecd-44ff-bcc7-d3d49af3eb86 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:45.593860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3531ef-4ecd-44ff-bcc7-d3d49af3eb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:45.612927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-de3531ef-4ecd-44ff-bcc7-d3d49af3eb86 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:45.614228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-de3531ef-4ecd-44ff-bcc7-d3d49af3eb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:45.621812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220de03-69e3-4fc4-b919-694e62c8afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:45.623001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220de03-69e3-4fc4-b919-694e62c8afd9 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:45.623514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220de03-69e3-4fc4-b919-694e62c8afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:45.624016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220de03-69e3-4fc4-b919-694e62c8afd9 None admin] PATH_INFO: `/v3/projects/26e616646e114f6f885b0d1df925801d/groups/8b2ffc8b2b08475dbc227a185caa690b/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:45.629851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220de03-69e3-4fc4-b919-694e62c8afd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=26e616646e114f6f885b0d1df925801d, group_id=8b2ffc8b2b08475dbc227a185caa690b, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:45.631550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220de03-69e3-4fc4-b919-694e62c8afd9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:45.686224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 218/416] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:45.709695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3531ef-4ecd-44ff-bcc7-d3d49af3eb86 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:45.725668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 199/417] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jun 16 00:16:45 2022] PUT /identity/v3/projects/26e616646e114f6f885b0d1df925801d/groups/8b2ffc8b2b08475dbc227a185caa690b/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:45.740941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:45.821779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3531ef-4ecd-44ff-bcc7-d3d49af3eb86 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:45.860565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928d135-bc35-4844-8b24-e76b2dee7ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:45.861382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928d135-bc35-4844-8b24-e76b2dee7ae6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:45.861671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928d135-bc35-4844-8b24-e76b2dee7ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:45.861958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928d135-bc35-4844-8b24-e76b2dee7ae6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:45.874363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928d135-bc35-4844-8b24-e76b2dee7ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-f928d135-bc35-4844-8b24-e76b2dee7ae6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:45.882331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:45.883310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:45.884171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 219/418] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:45.889656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220de03-69e3-4fc4-b919-694e62c8afd9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:46.028642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220de03-69e3-4fc4-b919-694e62c8afd9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:46.031311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a65b4d-ea7f-46fa-87e7-3490088dfc34 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:46.032341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a65b4d-ea7f-46fa-87e7-3490088dfc34 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:46.032341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a65b4d-ea7f-46fa-87e7-3490088dfc34 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:46.033585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 220/419] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:46.038659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335115d-3f27-4fe6-80a7-96e692cfc9f5 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:46.038954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335115d-3f27-4fe6-80a7-96e692cfc9f5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:46.039244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335115d-3f27-4fe6-80a7-96e692cfc9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:46.062814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5335115d-3f27-4fe6-80a7-96e692cfc9f5 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:46.063975 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5335115d-3f27-4fe6-80a7-96e692cfc9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:46.078503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:46.079169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:46.079463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:46.079747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:46.080514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] RBAC: Adding query filter params (name=member) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:46.080852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:46.082219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:46.099246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 200/420] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:46.105123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:46.122576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:46.224289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 221/421] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:46.244939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc849b3-0966-4e9d-aad4-03f042b6d47a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:46.279888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335115d-3f27-4fe6-80a7-96e692cfc9f5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:46.304904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:46.312141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebe7e1-dd68-4be4-9b29-26528bfeb56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:46.312981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebe7e1-dd68-4be4-9b29-26528bfeb56c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:46.324906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebe7e1-dd68-4be4-9b29-26528bfeb56c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:46.324906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebe7e1-dd68-4be4-9b29-26528bfeb56c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:46.388582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebe7e1-dd68-4be4-9b29-26528bfeb56c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:46.389977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebe7e1-dd68-4be4-9b29-26528bfeb56c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:46.412989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:46.488889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335115d-3f27-4fe6-80a7-96e692cfc9f5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:46.552394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 201/422] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:46.554038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c618b3f-4cdf-48be-b5f3-48dc1b385f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:46.554843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618b3f-4cdf-48be-b5f3-48dc1b385f90 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:46.555130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618b3f-4cdf-48be-b5f3-48dc1b385f90 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:46.555478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618b3f-4cdf-48be-b5f3-48dc1b385f90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:46.558076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebe7e1-dd68-4be4-9b29-26528bfeb56c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:46.580327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c618b3f-4cdf-48be-b5f3-48dc1b385f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-8c618b3f-4cdf-48be-b5f3-48dc1b385f90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:46.590589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:46.591592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:46.592407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 222/423] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:46.600225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c618b3f-4cdf-48be-b5f3-48dc1b385f90 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:46.734255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:46.757659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebe7e1-dd68-4be4-9b29-26528bfeb56c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:46.765313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c618b3f-4cdf-48be-b5f3-48dc1b385f90 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:46.798275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:46.805331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:46.806765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:46.807055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:46.807337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:46.808092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] RBAC: Adding query filter params (name=member) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:46.809389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:46.810022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:46.819045 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 223/424] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:16:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:46.836547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:46.845739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179d9e-2684-4dd0-8b7c-d07364b56d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:46.846817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179d9e-2684-4dd0-8b7c-d07364b56d1b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:46.847242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179d9e-2684-4dd0-8b7c-d07364b56d1b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:46.847659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179d9e-2684-4dd0-8b7c-d07364b56d1b None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:46.853465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f179d9e-2684-4dd0-8b7c-d07364b56d1b None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:46.855124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f179d9e-2684-4dd0-8b7c-d07364b56d1b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:46.857314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 202/425] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:46 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:46.862207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179d9e-2684-4dd0-8b7c-d07364b56d1b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:46.970422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74200600-0c43-4d58-b745-b99acfd5c30b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:47.002868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179d9e-2684-4dd0-8b7c-d07364b56d1b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:47.011376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad0451-c171-47c3-b749-1d697de0d0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:47.012032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad0451-c171-47c3-b749-1d697de0d0ad None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.012317 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad0451-c171-47c3-b749-1d697de0d0ad None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.012624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad0451-c171-47c3-b749-1d697de0d0ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.023211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:47.050373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61892e9-c9ad-4a9f-83b0-cfcc45ab2b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:47.051233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61892e9-c9ad-4a9f-83b0-cfcc45ab2b55 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.051688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61892e9-c9ad-4a9f-83b0-cfcc45ab2b55 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.052130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61892e9-c9ad-4a9f-83b0-cfcc45ab2b55 None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.053519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61892e9-c9ad-4a9f-83b0-cfcc45ab2b55 None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:47.056045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61892e9-c9ad-4a9f-83b0-cfcc45ab2b55 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:47.064229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 203/426] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:46 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:47.064229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13918ce-061f-47a6-bc14-e7d2b5e6c637 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.064229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13918ce-061f-47a6-bc14-e7d2b5e6c637 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.064229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13918ce-061f-47a6-bc14-e7d2b5e6c637 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.064229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 204/427] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:47.069293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13918ce-061f-47a6-bc14-e7d2b5e6c637 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:47.093564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad0451-c171-47c3-b749-1d697de0d0ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:47.095141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad0451-c171-47c3-b749-1d697de0d0ad None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:47.135867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:47.205750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 224/428] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:47.206511 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13918ce-061f-47a6-bc14-e7d2b5e6c637 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:47.207691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b495b2-e208-45bc-a6ab-223d3e23ebdb None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.208261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b495b2-e208-45bc-a6ab-223d3e23ebdb None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.208761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b495b2-e208-45bc-a6ab-223d3e23ebdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.226517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-66b495b2-e208-45bc-a6ab-223d3e23ebdb None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:47.228786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-66b495b2-e208-45bc-a6ab-223d3e23ebdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:47.247847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:47.249209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.249946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.250378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.251385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] RBAC: Adding query filter params (group.id=fee321acb6694877845fc83e036f9faf, role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=bcf793428063417f9cb53b3d0b4eea85) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:47.251853 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:47.254108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:47.269157 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 205/429] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:47 2022] GET /identity/v3/role_assignments?group.id=fee321acb6694877845fc83e036f9faf&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=64517449a51047a3b2ce1079b4912f57 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:47.269157 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:47.278868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:47.351034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 225/430] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:47.389421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b495b2-e208-45bc-a6ab-223d3e23ebdb None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:47.453501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58eef7-ce9c-441c-9528-f7ca755c6e84 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:47.486829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:47.535528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21131af-0151-42d6-a8f5-3a09162174f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:47.536678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21131af-0151-42d6-a8f5-3a09162174f2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.537123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21131af-0151-42d6-a8f5-3a09162174f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.537543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21131af-0151-42d6-a8f5-3a09162174f2 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.539032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21131af-0151-42d6-a8f5-3a09162174f2 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:47.541263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21131af-0151-42d6-a8f5-3a09162174f2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:47.543188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 206/431] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:47 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:47.549004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21131af-0151-42d6-a8f5-3a09162174f2 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:47.561913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b495b2-e208-45bc-a6ab-223d3e23ebdb None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:47.616471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91434-4cb4-4519-af92-b3824af5804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:47.617661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91434-4cb4-4519-af92-b3824af5804f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.618767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91434-4cb4-4519-af92-b3824af5804f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.619187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91434-4cb4-4519-af92-b3824af5804f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.630931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e91434-4cb4-4519-af92-b3824af5804f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-a8e91434-4cb4-4519-af92-b3824af5804f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:47.639883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:47.641030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:47.641938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:47.644012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 226/432] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:47.649821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91434-4cb4-4519-af92-b3824af5804f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:47.684669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:47.686956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21131af-0151-42d6-a8f5-3a09162174f2 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:47.721674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6c115-d87f-40b5-bdcd-904b6586588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:47.722804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6c115-d87f-40b5-bdcd-904b6586588d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.723232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6c115-d87f-40b5-bdcd-904b6586588d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.723711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6c115-d87f-40b5-bdcd-904b6586588d None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.725409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6c115-d87f-40b5-bdcd-904b6586588d None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:47.728323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6c115-d87f-40b5-bdcd-904b6586588d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:47.731102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 207/433] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:47 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:47.738899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6c115-d87f-40b5-bdcd-904b6586588d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:47.770639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91434-4cb4-4519-af92-b3824af5804f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:47.813032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:47.814185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.815096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.815543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.816516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:47.817005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:47.818874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:47.822087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:47.834236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 227/434] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:47.841656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:47.895780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6c115-d87f-40b5-bdcd-904b6586588d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:47.936887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6b431-0915-4686-bafd-b1acc55fc528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:47.937776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6b431-0915-4686-bafd-b1acc55fc528 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.938061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6b431-0915-4686-bafd-b1acc55fc528 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.938342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6b431-0915-4686-bafd-b1acc55fc528 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.939185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6b431-0915-4686-bafd-b1acc55fc528 None admin] RBAC: Adding query filter params (group.id=fee321acb6694877845fc83e036f9faf, role.id=64517449a51047a3b2ce1079b4912f57, scope.project.id=f03d23cb840648a38723008cbfdbbde7) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:47.939500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6b431-0915-4686-bafd-b1acc55fc528 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:47.940996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6b431-0915-4686-bafd-b1acc55fc528 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:47.948427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 208/435] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:47 2022] GET /identity/v3/role_assignments?group.id=fee321acb6694877845fc83e036f9faf&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=64517449a51047a3b2ce1079b4912f57 => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:47.950129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35dd14-3349-4737-beb9-4ea8e104055e None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.950616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35dd14-3349-4737-beb9-4ea8e104055e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.951107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35dd14-3349-4737-beb9-4ea8e104055e None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.953006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 209/436] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:47.958485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a230e-f950-4bc1-8134-ebf3b0e1d67a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:47.959249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a230e-f950-4bc1-8134-ebf3b0e1d67a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:47.959706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a230e-f950-4bc1-8134-ebf3b0e1d67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:47.978067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:47.980958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c89a230e-f950-4bc1-8134-ebf3b0e1d67a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:47.982935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c89a230e-f950-4bc1-8134-ebf3b0e1d67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:47.988829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632854f1-d7e5-4588-bb67-85b21b0e201b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:48.032851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247effbb-b445-4148-aa1f-d42436f720f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:48.033755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247effbb-b445-4148-aa1f-d42436f720f7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.034255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247effbb-b445-4148-aa1f-d42436f720f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.034535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247effbb-b445-4148-aa1f-d42436f720f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.083990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 210/437] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:48.111649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a230e-f950-4bc1-8134-ebf3b0e1d67a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.123111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247effbb-b445-4148-aa1f-d42436f720f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:48.125147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247effbb-b445-4148-aa1f-d42436f720f7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:48.175570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:48.243748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:48.253749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 228/438] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:16:48.253927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a230e-f950-4bc1-8134-ebf3b0e1d67a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:48.260497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247effbb-b445-4148-aa1f-d42436f720f7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.297432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c89616-03e0-4442-a2f9-76322ace090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:48.298910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c89616-03e0-4442-a2f9-76322ace090b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.298910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c89616-03e0-4442-a2f9-76322ace090b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.299092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c89616-03e0-4442-a2f9-76322ace090b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.310390 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c89616-03e0-4442-a2f9-76322ace090b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-f1c89616-03e0-4442-a2f9-76322ace090b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:48.322407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:48.323338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:48.324171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:48.325537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 211/439] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:48.330721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c89616-03e0-4442-a2f9-76322ace090b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.371394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247effbb-b445-4148-aa1f-d42436f720f7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:48.406132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca1a92-2051-49b3-8436-fdf6da59ba45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:48.406992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1a92-2051-49b3-8436-fdf6da59ba45 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.407277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1a92-2051-49b3-8436-fdf6da59ba45 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.407555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1a92-2051-49b3-8436-fdf6da59ba45 None admin] PATH_INFO: `/v3/projects/bcf793428063417f9cb53b3d0b4eea85` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.408772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca1a92-2051-49b3-8436-fdf6da59ba45 None admin] RBAC: Authorizing `identity:get_project(project_id=bcf793428063417f9cb53b3d0b4eea85)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:48.410201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca1a92-2051-49b3-8436-fdf6da59ba45 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:48.411869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 229/440] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/projects/bcf793428063417f9cb53b3d0b4eea85 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:48.413038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4cdea-0668-4bc0-aa8d-d73251fd51f8 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.413327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4cdea-0668-4bc0-aa8d-d73251fd51f8 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.413597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4cdea-0668-4bc0-aa8d-d73251fd51f8 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.415756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 230/441] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:48.419349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4cdea-0668-4bc0-aa8d-d73251fd51f8 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.434313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c89616-03e0-4442-a2f9-76322ace090b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:48.436611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:48.466850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bafc9e-3578-447a-b6bb-063fa5b9dda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:48.467501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafc9e-3578-447a-b6bb-063fa5b9dda6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.467782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafc9e-3578-447a-b6bb-063fa5b9dda6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.468056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafc9e-3578-447a-b6bb-063fa5b9dda6 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.468836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bafc9e-3578-447a-b6bb-063fa5b9dda6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:48.469138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bafc9e-3578-447a-b6bb-063fa5b9dda6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:48.470542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bafc9e-3578-447a-b6bb-063fa5b9dda6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:48.479279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:48.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 212/442] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:48.480668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4fc5d-f3e5-4017-9159-9cf0877aa281 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.480919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4fc5d-f3e5-4017-9159-9cf0877aa281 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.481195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4fc5d-f3e5-4017-9159-9cf0877aa281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.499275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f8a4fc5d-f3e5-4017-9159-9cf0877aa281 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:48.500466 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f8a4fc5d-f3e5-4017-9159-9cf0877aa281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:48.521784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4cdea-0668-4bc0-aa8d-d73251fd51f8 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:48.556944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57639b45-3b36-4a0b-85b9-22ec9f26801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:48.558112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57639b45-3b36-4a0b-85b9-22ec9f26801a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.558398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57639b45-3b36-4a0b-85b9-22ec9f26801a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.558676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57639b45-3b36-4a0b-85b9-22ec9f26801a None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.559868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57639b45-3b36-4a0b-85b9-22ec9f26801a None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:48.561306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57639b45-3b36-4a0b-85b9-22ec9f26801a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:48.562907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 231/443] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:48.563652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57639b45-3b36-4a0b-85b9-22ec9f26801a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.567260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 213/444] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:48.575089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4fc5d-f3e5-4017-9159-9cf0877aa281 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.666688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57639b45-3b36-4a0b-85b9-22ec9f26801a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:48.676385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:48.683319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4fc5d-f3e5-4017-9159-9cf0877aa281 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:48.692687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:48.700175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3625d-acb8-4afe-b555-917c82593dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:48.700854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3625d-acb8-4afe-b555-917c82593dca None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.701143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3625d-acb8-4afe-b555-917c82593dca None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.701423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3625d-acb8-4afe-b555-917c82593dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.716677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:48.717335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.719694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.719694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.719694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] RBAC: Adding query filter params (group.id=fee321acb6694877845fc83e036f9faf, role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=bcf793428063417f9cb53b3d0b4eea85) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:48.719694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:48.720404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:48.726153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 214/445] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/role_assignments?group.id=fee321acb6694877845fc83e036f9faf&scope.project.id=bcf793428063417f9cb53b3d0b4eea85&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:48.726908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.792315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3625d-acb8-4afe-b555-917c82593dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:48.793834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3625d-acb8-4afe-b555-917c82593dca None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:48.843866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792b501-ccb5-46bf-9f77-f243dd894a6b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:48.882041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7bf6-5c5e-4bc7-a863-5d6b25d1a364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:48.882840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7bf6-5c5e-4bc7-a863-5d6b25d1a364 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:48.883129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7bf6-5c5e-4bc7-a863-5d6b25d1a364 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:48.883444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7bf6-5c5e-4bc7-a863-5d6b25d1a364 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:48.895334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7bf6-5c5e-4bc7-a863-5d6b25d1a364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-042e7bf6-5c5e-4bc7-a863-5d6b25d1a364 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:48.903502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:48.904656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:48.905675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:48.906563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 215/446] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:48.907322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 232/447] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:48.912399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7bf6-5c5e-4bc7-a863-5d6b25d1a364 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.912653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3625d-acb8-4afe-b555-917c82593dca None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:48.938599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:48.945134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:49.022449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7bf6-5c5e-4bc7-a863-5d6b25d1a364 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:49.028878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3625d-acb8-4afe-b555-917c82593dca None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:49.063260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:49.063933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:49.064218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:49.064576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:49.065283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:49.066525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:49.066631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a9b2-f378-479f-8051-0bf889d44dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:49.066954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:49.067072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a9b2-f378-479f-8051-0bf889d44dd7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:49.067272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a9b2-f378-479f-8051-0bf889d44dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:49.067558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a9b2-f378-479f-8051-0bf889d44dd7 None admin] PATH_INFO: `/v3/projects/f03d23cb840648a38723008cbfdbbde7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:49.068846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256a9b2-f378-479f-8051-0bf889d44dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=f03d23cb840648a38723008cbfdbbde7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:49.070275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256a9b2-f378-479f-8051-0bf889d44dd7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:49.073280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 233/448] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/projects/f03d23cb840648a38723008cbfdbbde7 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:49.077080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 216/449] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:49.078223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a9b2-f378-479f-8051-0bf889d44dd7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:49.083223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:49.182242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:49.185423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:49.191216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a9b2-f378-479f-8051-0bf889d44dd7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:49.203934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f8db-8ddd-49f2-bd6c-215aa43c8616 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:49.228784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da23e-336a-4ba7-9698-d875cf6036d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:49.229965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da23e-336a-4ba7-9698-d875cf6036d3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:49.230416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da23e-336a-4ba7-9698-d875cf6036d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:49.230903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da23e-336a-4ba7-9698-d875cf6036d3 None admin] PATH_INFO: `/v3/groups/fee321acb6694877845fc83e036f9faf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:49.232408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da23e-336a-4ba7-9698-d875cf6036d3 None admin] RBAC: Authorizing `identity:get_group(group_id=fee321acb6694877845fc83e036f9faf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:49.234013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da23e-336a-4ba7-9698-d875cf6036d3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:49.236009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 234/450] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:49 2022] GET /identity/v3/groups/fee321acb6694877845fc83e036f9faf => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:49.242219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da23e-336a-4ba7-9698-d875cf6036d3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:49.244770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb357a7-db4c-4271-8418-8397cfe9fb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:49.245485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb357a7-db4c-4271-8418-8397cfe9fb89 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:49.245773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb357a7-db4c-4271-8418-8397cfe9fb89 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:49.246110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb357a7-db4c-4271-8418-8397cfe9fb89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:49.315379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb357a7-db4c-4271-8418-8397cfe9fb89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:49.316944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb357a7-db4c-4271-8418-8397cfe9fb89 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:49.342261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da23e-336a-4ba7-9698-d875cf6036d3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:49.375991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:49.376674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:49.376965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:49.377249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:49.379024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] RBAC: Adding query filter params (group.id=fee321acb6694877845fc83e036f9faf, role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=f03d23cb840648a38723008cbfdbbde7) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:49.379339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:49.380802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:49.386508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 235/451] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:49 2022] GET /identity/v3/role_assignments?group.id=fee321acb6694877845fc83e036f9faf&scope.project.id=f03d23cb840648a38723008cbfdbbde7&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:49.396942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:49.416848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 217/452] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:16:49.423171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:49.453807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:49.528589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a89b3-eb0c-4baf-a802-9198646f7e82 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:49.567415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8152e-0f6b-4004-b467-fca0bb8b04bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:49.568281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8152e-0f6b-4004-b467-fca0bb8b04bc None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:49.568595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8152e-0f6b-4004-b467-fca0bb8b04bc None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:49.568883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8152e-0f6b-4004-b467-fca0bb8b04bc None admin] PATH_INFO: `/v3/projects/26e616646e114f6f885b0d1df925801d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:49.570079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8152e-0f6b-4004-b467-fca0bb8b04bc None admin] RBAC: Authorizing `identity:get_project(project_id=26e616646e114f6f885b0d1df925801d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:49.571564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8152e-0f6b-4004-b467-fca0bb8b04bc None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:49.573285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 236/453] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:49 2022] GET /identity/v3/projects/26e616646e114f6f885b0d1df925801d => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:49.578826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb357a7-db4c-4271-8418-8397cfe9fb89 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:49.688594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb357a7-db4c-4271-8418-8397cfe9fb89 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:49.722990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ac16b-92b2-45fc-ad51-6d3b3531f2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:49.723771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ac16b-92b2-45fc-ad51-6d3b3531f2e2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:49.724052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ac16b-92b2-45fc-ad51-6d3b3531f2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:49.724322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ac16b-92b2-45fc-ad51-6d3b3531f2e2 None admin] PATH_INFO: `/v3/groups/8b2ffc8b2b08475dbc227a185caa690b` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:49.725562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ac16b-92b2-45fc-ad51-6d3b3531f2e2 None admin] RBAC: Authorizing `identity:get_group(group_id=8b2ffc8b2b08475dbc227a185caa690b)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:49.726989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ac16b-92b2-45fc-ad51-6d3b3531f2e2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:49.729367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 218/454] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jun 16 00:16:49 2022] GET /identity/v3/groups/8b2ffc8b2b08475dbc227a185caa690b => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:49.735225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8152e-0f6b-4004-b467-fca0bb8b04bc None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:49.830342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:49.835818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8152e-0f6b-4004-b467-fca0bb8b04bc None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:49.868497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1626a84-cb07-46ef-9d29-2c74eb7628a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:49.869169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1626a84-cb07-46ef-9d29-2c74eb7628a4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:49.869455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1626a84-cb07-46ef-9d29-2c74eb7628a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:49.869736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1626a84-cb07-46ef-9d29-2c74eb7628a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:49.870566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1626a84-cb07-46ef-9d29-2c74eb7628a4 None admin] RBAC: Adding query filter params (group.id=8b2ffc8b2b08475dbc227a185caa690b, role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=26e616646e114f6f885b0d1df925801d) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:49.870887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1626a84-cb07-46ef-9d29-2c74eb7628a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:49.872302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1626a84-cb07-46ef-9d29-2c74eb7628a4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:49.877966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 237/455] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jun 16 00:16:49 2022] GET /identity/v3/role_assignments?group.id=8b2ffc8b2b08475dbc227a185caa690b&scope.project.id=26e616646e114f6f885b0d1df925801d&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:52.398919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ea1c-ff3a-4c40-98af-c9ae6589d281 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:52.399500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ea1c-ff3a-4c40-98af-c9ae6589d281 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:52.399500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ea1c-ff3a-4c40-98af-c9ae6589d281 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:52.401431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 219/456] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:52.406513 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ba33-7dbf-44d6-9ca7-3921a7d6edf8 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:52.406898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ba33-7dbf-44d6-9ca7-3921a7d6edf8 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:52.407078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ba33-7dbf-44d6-9ca7-3921a7d6edf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:52.429482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f8c4ba33-7dbf-44d6-9ca7-3921a7d6edf8 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:52.431229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f8c4ba33-7dbf-44d6-9ca7-3921a7d6edf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:52.523084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 238/457] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:52.556504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2ea1c-ff3a-4c40-98af-c9ae6589d281 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:52.615912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a150618-b2d9-48fa-a5bf-361a1ccfd552 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:52.616196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a150618-b2d9-48fa-a5bf-361a1ccfd552 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:52.616474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a150618-b2d9-48fa-a5bf-361a1ccfd552 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:52.617891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 239/458] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:52.623175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79867d70-5f26-4a19-93e6-e8acd6af824b None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:52.623467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79867d70-5f26-4a19-93e6-e8acd6af824b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:52.623758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79867d70-5f26-4a19-93e6-e8acd6af824b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:52.648671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-79867d70-5f26-4a19-93e6-e8acd6af824b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:52.648857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:52.650842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-79867d70-5f26-4a19-93e6-e8acd6af824b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:52.755031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2ea1c-ff3a-4c40-98af-c9ae6589d281 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:52.758870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 240/459] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:52.783506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79867d70-5f26-4a19-93e6-e8acd6af824b None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:52.821478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:52.835197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c365ce-a495-40c0-86c6-112f72cb3bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:52.836002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c365ce-a495-40c0-86c6-112f72cb3bd3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:52.836286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c365ce-a495-40c0-86c6-112f72cb3bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:52.836616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c365ce-a495-40c0-86c6-112f72cb3bd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:52.860491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c365ce-a495-40c0-86c6-112f72cb3bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:52.862006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c365ce-a495-40c0-86c6-112f72cb3bd3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-c2c365ce-a495-40c0-86c6-112f72cb3bd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:52.880446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:52.881379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:52.882277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:52.882277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:52.882277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:52.882277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:52.882277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 220/460] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:52.883367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165f4bb-8310-47ff-9354-6f62d43b7062 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:52.883752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165f4bb-8310-47ff-9354-6f62d43b7062 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:52.884006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165f4bb-8310-47ff-9354-6f62d43b7062 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:52.885307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 221/461] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:52.887391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165f4bb-8310-47ff-9354-6f62d43b7062 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:52.932820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79867d70-5f26-4a19-93e6-e8acd6af824b None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:52.983857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c0294-9c85-40c4-be88-bcfdab2ca239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:52.984939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c0294-9c85-40c4-be88-bcfdab2ca239 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:52.985616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c0294-9c85-40c4-be88-bcfdab2ca239 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:52.985904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c0294-9c85-40c4-be88-bcfdab2ca239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.000485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c0294-9c85-40c4-be88-bcfdab2ca239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.002065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c0294-9c85-40c4-be88-bcfdab2ca239 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.002575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165f4bb-8310-47ff-9354-6f62d43b7062 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:53.004343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-8f5c0294-9c85-40c4-be88-bcfdab2ca239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:53.025046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:53.025992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:53.026949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:53.026949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:53.026949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:53.026949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:53.027621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 241/462] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:53.028911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38299-08c2-4012-a152-137c96f2941a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.031156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38299-08c2-4012-a152-137c96f2941a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.031156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38299-08c2-4012-a152-137c96f2941a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.046336 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:53.047004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.047292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.047575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.048661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:53.048974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.050451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.050628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5dc38299-08c2-4012-a152-137c96f2941a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:53.051711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5dc38299-08c2-4012-a152-137c96f2941a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:53.063592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 222/463] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:53.064679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:53.143308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 242/464] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:53.144891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc38299-08c2-4012-a152-137c96f2941a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:53.147585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:53.183481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f809-23bd-4b4a-80bc-78508035bdf9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:53.220849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:53.221643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.221920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.222260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.223013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:53.223319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.224713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.237501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 223/465] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:53.238760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:53.264247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:53.270896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc38299-08c2-4012-a152-137c96f2941a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:53.305924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7505e-054d-4df8-a905-c442d11992a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:53.306603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7505e-054d-4df8-a905-c442d11992a2 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.306889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7505e-054d-4df8-a905-c442d11992a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.307168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7505e-054d-4df8-a905-c442d11992a2 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.307866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7505e-054d-4df8-a905-c442d11992a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.309344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7505e-054d-4df8-a905-c442d11992a2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.345473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df7189-a143-45e4-88de-9b8cad04c592 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:53.379652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682d578-f45c-4e5c-ad00-366076754d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:53.380435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d578-f45c-4e5c-ad00-366076754d38 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.380746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d578-f45c-4e5c-ad00-366076754d38 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.381026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d578-f45c-4e5c-ad00-366076754d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.391277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682d578-f45c-4e5c-ad00-366076754d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.392743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682d578-f45c-4e5c-ad00-366076754d38 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-0682d578-f45c-4e5c-ad00-366076754d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:53.412260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:53.413784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:53.415384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:53.415384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:53.415384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:53.415384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:53.415384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 224/466] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:53.415384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ec09d-3023-4785-ac53-b51277fbdd51 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.415785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ec09d-3023-4785-ac53-b51277fbdd51 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.415785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ec09d-3023-4785-ac53-b51277fbdd51 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.424568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 225/467] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:53.424568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ec09d-3023-4785-ac53-b51277fbdd51 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:53.432647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 243/468] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:53 2022] POST /identity/v3/users => generated 246 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:53.432647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742726c-c7b2-410d-a800-af6f3ba73d4d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.432647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742726c-c7b2-410d-a800-af6f3ba73d4d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.432647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742726c-c7b2-410d-a800-af6f3ba73d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.447035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-0742726c-c7b2-410d-a800-af6f3ba73d4d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:53.448190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0742726c-c7b2-410d-a800-af6f3ba73d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:53.512265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:53.518398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:53.529568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 244/469] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:53.530859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742726c-c7b2-410d-a800-af6f3ba73d4d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:53.567659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ec09d-3023-4785-ac53-b51277fbdd51 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:53.617553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7860a7-c8b3-4ff1-b499-ad8353e4cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:53.620614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7860a7-c8b3-4ff1-b499-ad8353e4cc8a None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.621637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7860a7-c8b3-4ff1-b499-ad8353e4cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.622062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7860a7-c8b3-4ff1-b499-ad8353e4cc8a None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.624686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7860a7-c8b3-4ff1-b499-ad8353e4cc8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.626284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7860a7-c8b3-4ff1-b499-ad8353e4cc8a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.660788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742726c-c7b2-410d-a800-af6f3ba73d4d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:53.698088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:53.698994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.699424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.699857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.700633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:53.700947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.702315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.712294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 226/470] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:53 2022] POST /identity/v3/users => generated 248 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:53.713190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7860a7-c8b3-4ff1-b499-ad8353e4cc8a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:53.720084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 245/471] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:53.731862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:53.834284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:53.916009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7860a7-c8b3-4ff1-b499-ad8353e4cc8a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:53.933863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd91fe-e78d-42bb-887d-ce9514c9bf30 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:53.949436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:53.956307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ff00a-542d-4896-82ad-6a45e787bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:53.957151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff00a-542d-4896-82ad-6a45e787bc70 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.957433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff00a-542d-4896-82ad-6a45e787bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.957711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff00a-542d-4896-82ad-6a45e787bc70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.971649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ff00a-542d-4896-82ad-6a45e787bc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.973159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ff00a-542d-4896-82ad-6a45e787bc70 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.980313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39a037-c0c5-4836-ac87-2195dd742aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:53.981082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39a037-c0c5-4836-ac87-2195dd742aa7 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:53.981873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39a037-c0c5-4836-ac87-2195dd742aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:53.981873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39a037-c0c5-4836-ac87-2195dd742aa7 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:53.982347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39a037-c0c5-4836-ac87-2195dd742aa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:53.984662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39a037-c0c5-4836-ac87-2195dd742aa7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-f09ff00a-542d-4896-82ad-6a45e787bc70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:53.999103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:54.000353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:54.001294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:54.001294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:54.001294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:54.001294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:54.001294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 227/472] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:54.001974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385286e2-31d0-4b75-9670-cecfedc19179 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:54.002070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385286e2-31d0-4b75-9670-cecfedc19179 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:54.002498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385286e2-31d0-4b75-9670-cecfedc19179 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:54.003743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 228/473] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:54.006659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385286e2-31d0-4b75-9670-cecfedc19179 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:54.073346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 246/474] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:53 2022] POST /identity/v3/users => generated 248 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:54.075447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b9d83-ab35-4dcf-9594-274a46e9c504 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:54.076011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b9d83-ab35-4dcf-9594-274a46e9c504 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:54.076352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b9d83-ab35-4dcf-9594-274a46e9c504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:54.103244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-0c4b9d83-ab35-4dcf-9594-274a46e9c504 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:54.104847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0c4b9d83-ab35-4dcf-9594-274a46e9c504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:54.131393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385286e2-31d0-4b75-9670-cecfedc19179 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:54.176362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:54.177074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:54.177363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:54.177647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:54.178443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:54.178755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:54.181551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:54.197391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 247/475] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:54.198326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 229/476] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:54.210412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b9d83-ab35-4dcf-9594-274a46e9c504 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:54.248744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:54.274904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:54.325549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:54.366492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b9d83-ab35-4dcf-9594-274a46e9c504 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:54.393343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6167c-e3df-4609-903c-4547d1fe9dc1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:54.403880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a82f19-fb40-416f-9873-a256ce638ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:54.404598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a82f19-fb40-416f-9873-a256ce638ce9 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:54.404891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a82f19-fb40-416f-9873-a256ce638ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:54.405173 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a82f19-fb40-416f-9873-a256ce638ce9 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:54.405900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a82f19-fb40-416f-9873-a256ce638ce9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:54.407541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a82f19-fb40-416f-9873-a256ce638ce9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:54.435149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:54.435831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:54.436115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:54.437044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:54.437265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:54.438574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:54.482473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.sql.core [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 00:16:54.482473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 00:16:54.482473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [parameters: {'id': '5c7456e9dd224f88b933a4e20ec3cb92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 00:16:54.482473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86685) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application result.read() Jun 16 00:16:54.500197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:16:54.501165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application yield resource Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise self.value Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application session.commit() Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 00:16:54.502093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self.session.flush() Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application compat.raise_( Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 00:16:54.503015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:16:54.503924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application result.read() Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:16:54.506023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [parameters: {'id': '5c7456e9dd224f88b933a4e20ec3cb92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:54.507981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 00:16:54.510113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:54.511643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 230/477] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jun 16 00:16:54 2022] POST /identity/v3/roles => generated 144 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:54.511643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:54.530503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 248/478] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:54 2022] POST /identity/v3/users => generated 249 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:54.639218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1e881-ed2f-406a-80ad-2591e65ddcde None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:54.647257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:54.676555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da927-39fd-4c59-b320-8c8a377c87ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:54.677458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da927-39fd-4c59-b320-8c8a377c87ed None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:54.677888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da927-39fd-4c59-b320-8c8a377c87ed None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:54.678173 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da927-39fd-4c59-b320-8c8a377c87ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:54.691187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5da927-39fd-4c59-b320-8c8a377c87ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-ea5da927-39fd-4c59-b320-8c8a377c87ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:54.700759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:54.706154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:54.706980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 231/479] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:16:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:54.709156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a82f19-fb40-416f-9873-a256ce638ce9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:54.880309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:54.889860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a82f19-fb40-416f-9873-a256ce638ce9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:54.929772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58105f22-f6a3-4c19-b4ca-e8eb1e6f9cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:54.930478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58105f22-f6a3-4c19-b4ca-e8eb1e6f9cf8 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:54.930765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58105f22-f6a3-4c19-b4ca-e8eb1e6f9cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:54.931044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58105f22-f6a3-4c19-b4ca-e8eb1e6f9cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:54.931802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58105f22-f6a3-4c19-b4ca-e8eb1e6f9cf8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:54.932111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58105f22-f6a3-4c19-b4ca-e8eb1e6f9cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:54.933584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58105f22-f6a3-4c19-b4ca-e8eb1e6f9cf8 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:54.943250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 249/480] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:55.981524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f19d76-7ea7-47e0-b09c-771775d6a152 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:55.982010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f19d76-7ea7-47e0-b09c-771775d6a152 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:55.982010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f19d76-7ea7-47e0-b09c-771775d6a152 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:55.983624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 232/481] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:55.994263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05002f15-117d-4ba9-80c2-aacbc3cb4131 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:55.994681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05002f15-117d-4ba9-80c2-aacbc3cb4131 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:55.994857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05002f15-117d-4ba9-80c2-aacbc3cb4131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:56.021521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-05002f15-117d-4ba9-80c2-aacbc3cb4131 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:56.022729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-05002f15-117d-4ba9-80c2-aacbc3cb4131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:56.110123 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 250/482] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:56.139538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f19d76-7ea7-47e0-b09c-771775d6a152 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:56.281908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f19d76-7ea7-47e0-b09c-771775d6a152 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:56.322591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7dc25-6d80-4b11-b9a9-58c4bd1be93c None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:56.322875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7dc25-6d80-4b11-b9a9-58c4bd1be93c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:56.323161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7dc25-6d80-4b11-b9a9-58c4bd1be93c None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:56.327062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d02e-8957-4565-bfe7-a20d278fe8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:56.327860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d02e-8957-4565-bfe7-a20d278fe8ae None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:56.328144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d02e-8957-4565-bfe7-a20d278fe8ae None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:56.328386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 251/483] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:56.328484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d02e-8957-4565-bfe7-a20d278fe8ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:56.329866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652bc8-8757-4da3-ab92-e78585382788 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:56.330162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652bc8-8757-4da3-ab92-e78585382788 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:56.330446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652bc8-8757-4da3-ab92-e78585382788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:56.355730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-dc652bc8-8757-4da3-ab92-e78585382788 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:56.356940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dc652bc8-8757-4da3-ab92-e78585382788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:56.358706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4d02e-8957-4565-bfe7-a20d278fe8ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-fce4d02e-8957-4565-bfe7-a20d278fe8ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:56.366310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:56.368773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:16:56.370817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:56.382197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 233/484] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:16:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:56.382197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d02e-8957-4565-bfe7-a20d278fe8ae None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:56.437427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:56.488359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 252/485] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:16:56.520014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652bc8-8757-4da3-ab92-e78585382788 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:56.538746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:56.553163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d02e-8957-4565-bfe7-a20d278fe8ae None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:56.609767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:56.610446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:56.610739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:56.611019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:56.611774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:56.612083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:56.613469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:56.630764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 234/486] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:16:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:56.636143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:56.659596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652bc8-8757-4da3-ab92-e78585382788 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:56.707760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb800e1-4fff-48af-bfd7-5ebeca454f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:56.708656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb800e1-4fff-48af-bfd7-5ebeca454f0e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:56.708952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb800e1-4fff-48af-bfd7-5ebeca454f0e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:56.709228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb800e1-4fff-48af-bfd7-5ebeca454f0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:56.727611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb800e1-4fff-48af-bfd7-5ebeca454f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-4fb800e1-4fff-48af-bfd7-5ebeca454f0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:56.742234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:56.743788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:56.744645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 253/487] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:16:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:56.751660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb800e1-4fff-48af-bfd7-5ebeca454f0e None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:56.759069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:56.773510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f641d-ace2-49e0-b9bf-e79c874ef2a1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:56.815525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:56.816233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:56.816514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:56.816825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:56.843295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:56.845542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:56.868397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:56.869361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:56.870226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:56.870226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:56.870226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:56.870226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:56.870226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 235/488] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:16:56.875380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:56.905401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:56.940556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb800e1-4fff-48af-bfd7-5ebeca454f0e None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:56.992950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004b71c-aa3d-4450-a372-05b2717db4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:56.993737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004b71c-aa3d-4450-a372-05b2717db4e5 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:56.994021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004b71c-aa3d-4450-a372-05b2717db4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:56.994303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004b71c-aa3d-4450-a372-05b2717db4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:56.995056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004b71c-aa3d-4450-a372-05b2717db4e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:56.995369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004b71c-aa3d-4450-a372-05b2717db4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:56.996981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004b71c-aa3d-4450-a372-05b2717db4e5 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.010079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 254/489] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:16:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:57.010079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21910-ffd1-4b45-80da-16dde98bc7f0 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.010079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21910-ffd1-4b45-80da-16dde98bc7f0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.010079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21910-ffd1-4b45-80da-16dde98bc7f0 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.010079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 255/490] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:57.013465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de21910-ffd1-4b45-80da-16dde98bc7f0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.043280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb791a-7f8c-4d15-965a-c172afcac1ca None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.090001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483221e8-25a7-4d49-8f0b-9dc935851263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.091203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483221e8-25a7-4d49-8f0b-9dc935851263 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.092558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483221e8-25a7-4d49-8f0b-9dc935851263 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.093029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483221e8-25a7-4d49-8f0b-9dc935851263 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.093939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483221e8-25a7-4d49-8f0b-9dc935851263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:57.094425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483221e8-25a7-4d49-8f0b-9dc935851263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:57.095992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483221e8-25a7-4d49-8f0b-9dc935851263 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.108472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:57.112344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 236/491] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:57.114540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2efd4b-fc2a-437d-9f4a-a4b5fdd128a8 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.115018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2efd4b-fc2a-437d-9f4a-a4b5fdd128a8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.115462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2efd4b-fc2a-437d-9f4a-a4b5fdd128a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.127285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de21910-ffd1-4b45-80da-16dde98bc7f0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.136788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8c2efd4b-fc2a-437d-9f4a-a4b5fdd128a8 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:57.138703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8c2efd4b-fc2a-437d-9f4a-a4b5fdd128a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:57.177306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.179299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.179796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.180284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.182520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:57.194708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:57.196498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:57.215997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:57.216929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:57.217877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:57.217877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:57.217877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:57.217877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:57.224287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 237/493] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:57.224287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2efd4b-fc2a-437d-9f4a-a4b5fdd128a8 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.224720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 256/493] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:57.225880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.340517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e436677-4d03-4ffc-ad44-190aa78f58c5 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.346491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2efd4b-fc2a-437d-9f4a-a4b5fdd128a8 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.375757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:57.377710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.378297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.378600 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.378859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.379581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:57.379893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:57.381327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.381912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46342419-1162-46a3-b158-e1f0f21653cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.382705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342419-1162-46a3-b158-e1f0f21653cf None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.382993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342419-1162-46a3-b158-e1f0f21653cf None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.383807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342419-1162-46a3-b158-e1f0f21653cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.391385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46342419-1162-46a3-b158-e1f0f21653cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:57.392841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46342419-1162-46a3-b158-e1f0f21653cf None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.395035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 257/494] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:57.399665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-46342419-1162-46a3-b158-e1f0f21653cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:16:57.413876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:57.415085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:57.416181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 238/495] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:57.416924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6498a51-354e-4a72-a392-66701e8e156c None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.417006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6498a51-354e-4a72-a392-66701e8e156c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.417249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6498a51-354e-4a72-a392-66701e8e156c None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.418846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 239/496] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:57.419624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6498a51-354e-4a72-a392-66701e8e156c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.454560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:57.516264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3e05e-0844-46e3-94ae-e21482b490cf None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.529869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6498a51-354e-4a72-a392-66701e8e156c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.552621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98cf04-c11e-4aca-8b1d-d4c6ad6291c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.553426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98cf04-c11e-4aca-8b1d-d4c6ad6291c7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.553726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98cf04-c11e-4aca-8b1d-d4c6ad6291c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.553994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98cf04-c11e-4aca-8b1d-d4c6ad6291c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.564911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98cf04-c11e-4aca-8b1d-d4c6ad6291c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:57.566284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d57554-59d9-4363-bd9e-16566531ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.567078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d57554-59d9-4363-bd9e-16566531ad0d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.567370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d57554-59d9-4363-bd9e-16566531ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.567655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d57554-59d9-4363-bd9e-16566531ad0d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-3f98cf04-c11e-4aca-8b1d-d4c6ad6291c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:57.572565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:57.573474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 258/497] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:57.579451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98cf04-c11e-4aca-8b1d-d4c6ad6291c7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.583642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d57554-59d9-4363-bd9e-16566531ad0d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:57.585143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d57554-59d9-4363-bd9e-16566531ad0d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-52d57554-59d9-4363-bd9e-16566531ad0d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:16:57.600613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:57.601710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:57.603207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 240/498] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:57.603798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543ce96-2ac6-4371-9964-d14189622c99 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.603864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543ce96-2ac6-4371-9964-d14189622c99 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.604464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543ce96-2ac6-4371-9964-d14189622c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.621186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7543ce96-2ac6-4371-9964-d14189622c99 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:57.622371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7543ce96-2ac6-4371-9964-d14189622c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:57.671776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:57.696668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 241/499] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:57.697730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543ce96-2ac6-4371-9964-d14189622c99 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.714591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98cf04-c11e-4aca-8b1d-d4c6ad6291c7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.726231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:57.754465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.755154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.755443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.755728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.756598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:57.756918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:57.758398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.796895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 259/500] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:57.797813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.812556 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543ce96-2ac6-4371-9964-d14189622c99 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.854981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.855571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.855858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.856142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.856927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:57.857238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:57.858598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.866654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 242/501] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:57.867451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.905909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a1dc6-57fa-4772-b2b4-8ffd33b31e5a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.909691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:57.943660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6b19-eaf7-4f06-9237-bb3568e2813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:57.944325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6b19-eaf7-4f06-9237-bb3568e2813b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.944638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6b19-eaf7-4f06-9237-bb3568e2813b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.944916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6b19-eaf7-4f06-9237-bb3568e2813b None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.945765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db6b19-eaf7-4f06-9237-bb3568e2813b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:57.946634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db6b19-eaf7-4f06-9237-bb3568e2813b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:57.948049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db6b19-eaf7-4f06-9237-bb3568e2813b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:57.964368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 260/502] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:57.964819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2316a8-2aca-4517-accf-9155b522e4e6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:57.965611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90154513-cb7c-4713-a68c-96034ebadf8a None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:57.965900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90154513-cb7c-4713-a68c-96034ebadf8a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:57.966179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90154513-cb7c-4713-a68c-96034ebadf8a None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:57.967390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 261/503] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:16:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:57.968079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90154513-cb7c-4713-a68c-96034ebadf8a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:57.991120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.001861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b71a2d-8fbd-4f3c-aa80-7ae4de8424ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.002652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71a2d-8fbd-4f3c-aa80-7ae4de8424ad None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.002939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71a2d-8fbd-4f3c-aa80-7ae4de8424ad None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.003207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71a2d-8fbd-4f3c-aa80-7ae4de8424ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.013613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b71a2d-8fbd-4f3c-aa80-7ae4de8424ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d6b71a2d-8fbd-4f3c-aa80-7ae4de8424ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:58.021134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:58.022040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:16:58.022893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:58.024012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 243/504] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:58.024977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b71a2d-8fbd-4f3c-aa80-7ae4de8424ad None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.066989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90154513-cb7c-4713-a68c-96034ebadf8a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.100350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a659bb-6409-4cce-b058-f7d656db9732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.101024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a659bb-6409-4cce-b058-f7d656db9732 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.101308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a659bb-6409-4cce-b058-f7d656db9732 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.101583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a659bb-6409-4cce-b058-f7d656db9732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.120940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b71a2d-8fbd-4f3c-aa80-7ae4de8424ad None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.122378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.153673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.154502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.155203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.155339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.165956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:58.167385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:58.167603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a659bb-6409-4cce-b058-f7d656db9732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:58.169065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a659bb-6409-4cce-b058-f7d656db9732 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:58.185242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:58.186206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:58.187082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:58.187082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:58.187082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:58.187082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:58.187082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 244/505] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:58.187349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.193955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.268753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 262/506] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:58.270160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ce302-005b-4c32-8434-5ae582684314 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.270453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ce302-005b-4c32-8434-5ae582684314 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.270730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ce302-005b-4c32-8434-5ae582684314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.282868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdf740-f896-48a8-80f7-072b5f7243f8 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.286051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-9c7ce302-005b-4c32-8434-5ae582684314 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:58.287233 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9c7ce302-005b-4c32-8434-5ae582684314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:58.317444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff231ad-2f44-4dca-9187-405a52cea40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.318082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff231ad-2f44-4dca-9187-405a52cea40d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.318365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff231ad-2f44-4dca-9187-405a52cea40d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.318644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff231ad-2f44-4dca-9187-405a52cea40d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.341246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.350951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 263/507] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:16:58.351683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ce302-005b-4c32-8434-5ae582684314 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.387675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff231ad-2f44-4dca-9187-405a52cea40d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:58.389085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff231ad-2f44-4dca-9187-405a52cea40d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:58.404628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.450838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ce302-005b-4c32-8434-5ae582684314 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.484315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.485019 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.485304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.485518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 245/508] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:16:58.485619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.486338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:58.486435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff231ad-2f44-4dca-9187-405a52cea40d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.486609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:58.487980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:58.496200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 264/509] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:58.496885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.542804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.587932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff231ad-2f44-4dca-9187-405a52cea40d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.596315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028af2-3f64-4677-b04f-e8e46a8c59e2 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.610503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.620688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.621344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.629978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.629978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.629978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:58.629978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:58.629978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:58.630310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.630310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.630629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.630828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.635965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 246/510] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:58.636769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.641262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:58.642743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:58.660336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:58.661247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:58.662118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:58.662118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:58.662118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:58.662118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:58.662118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 265/511] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:58.662393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.791810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b071608-b78d-4ff0-a62c-b2a71bdfe824 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.805808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.825491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.826277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.826564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.826840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.826929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d911c-9440-45fb-9fa0-d08557dfccb7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.837375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:58.839116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:58.856903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:58.857748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:58.858628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:58.858628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:58.858628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:58.858628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:58.858628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 247/512] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:58.858900 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.864839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:58.865639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:58.865928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:58.866212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:58.877254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:58.878687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:58.895604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:58.896583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:58.897449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:58.897449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:58.897449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:58.897449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:58.897449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 266/513] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:58.897721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:58.916281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:58.966340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0993290-4f51-492e-aa42-d868e8a623c9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:58.996434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006b45-0b08-4996-96e5-98f018d5a853 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.000629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.001412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.001693 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.001966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.011670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.013145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.029779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:59.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:59.031189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:59.032091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:16:59.032091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:59.032091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:16:59.032091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:16:59.032314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.032314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.032314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] PATH_INFO: `/v3/users/glance` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.032496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 248/514] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:59.033480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.038587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.040567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.051698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:16:59.056873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:59.057793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:59.058691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 267/515] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:59.059261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.134633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762fdb2-9f22-417c-ad3c-3a2bc2717475 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.135148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:59.156671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1904365-5be3-4920-90af-58d4d3042b2d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.168688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.169348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.169631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.169917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.170640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:59.170949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.172324 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.184058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 249/516] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:59.184858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.191353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.192016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.192300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.192614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.193332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:59.193638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.195005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.207353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 268/517] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:59.208135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.267118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:59.288080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdcd1e-205f-422e-aeeb-2cbcb57e9cff None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.304872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8cd65-f74b-4330-b45c-6e0397e40328 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.321394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.322047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.322328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.322604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.323320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:59.323622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.325066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.337012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 250/518] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:59.337822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.342805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.343464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.343745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.344024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.344810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:59.345122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.346488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.353910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:59.363602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 269/519] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:59.363602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.438274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8181d-9b35-4e9c-a08c-921eec40c8c1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.455956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2fdef-f871-4b3e-a294-8b704364c906 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.471319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.471982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.472268 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.472573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.473412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:59.473787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.475235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.490136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.490925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.491217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.491495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.492999 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:59.499127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.500591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.511984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 251/520] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:59.521297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:16:59.521434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:59.522239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 270/521] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:59.523106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.575355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:59.613603 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184a5a8-ea9f-4763-8bc6-438d12ead0fc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.616616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621d15c-1a7b-4c6f-8a59-b3e6772ad3de None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.646323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9037f3-808c-4c5b-9d68-1ec6d6c5cfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.646989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9037f3-808c-4c5b-9d68-1ec6d6c5cfb1 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.647274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9037f3-808c-4c5b-9d68-1ec6d6c5cfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.647552 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9037f3-808c-4c5b-9d68-1ec6d6c5cfb1 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.648255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9037f3-808c-4c5b-9d68-1ec6d6c5cfb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.648504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.649849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.650129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.650286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9037f3-808c-4c5b-9d68-1ec6d6c5cfb1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.650403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.657221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.658627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:16:59.672871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:59.676585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 271/522] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:59.677662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.779102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e499a-806e-4517-8945-7a96ac7c8e10 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.790644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 252/523] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:59 2022] POST /identity/v3/users => generated 247 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:59.791579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9037f3-808c-4c5b-9d68-1ec6d6c5cfb1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.803061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:59.815800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.816665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.816985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.817265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.826010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:59.827508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:16:59.847893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:59.848969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:16:59.849894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:16:59.853331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 272/524] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:59.858999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:59.864228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:59.918159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9037f3-808c-4c5b-9d68-1ec6d6c5cfb1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:59.963982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93e3bf-5d1a-4f9e-8a2b-3f90c59a6214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:59.964434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e3bf-5d1a-4f9e-8a2b-3f90c59a6214 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:59.964752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e3bf-5d1a-4f9e-8a2b-3f90c59a6214 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:59.965662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e3bf-5d1a-4f9e-8a2b-3f90c59a6214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:59.974910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2e399-72d7-43da-a9b3-74913a760dad None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:00.018511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:00.019181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:00.019469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:00.019748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:00.020556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:00.020870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.022310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.039691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 273/525] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:00.040558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:00.049433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93e3bf-5d1a-4f9e-8a2b-3f90c59a6214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.050977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93e3bf-5d1a-4f9e-8a2b-3f90c59a6214 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.088248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:00.118149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:00.151290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d807994-85d6-44c7-88b6-504f56585104 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:00.165638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 253/526] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:00.166581 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93e3bf-5d1a-4f9e-8a2b-3f90c59a6214 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:00.193926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:00.194593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:00.194880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:00.195157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:00.196497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:00.196837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.198274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.214807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 274/527] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:00.214807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:00.280908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93e3bf-5d1a-4f9e-8a2b-3f90c59a6214 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:00.310025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:00.314306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c49efc-584e-4d6e-9826-840fc57d0323 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:00.318547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:00.319218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:00.319501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:00.319773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:00.320717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:00.321594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.323015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.342017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 254/528] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:00.342664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:00.354813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:00.355006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:00.355471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:00.355755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:00.356036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:00.356901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=dea47e0ec737422c8f679e21174f8cab) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:00.357224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.358665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.372566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 275/529] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/role_assignments?user.id=dea47e0ec737422c8f679e21174f8cab&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:00.372566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:00.480145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b044e-2733-4ff2-bc34-ddde18fb70d1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:00.493444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a27c2-4deb-440b-a4d9-91514bceb55f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:00.534385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:00.535170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:00.535477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:00.535721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:00.536298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:00.537095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:00.537480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:00.537878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:00.538836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=fff82e4b1a4248fcbb3566bb7bc84d26) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:00.539258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.541445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.548457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 276/530] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/role_assignments?user.id=fff82e4b1a4248fcbb3566bb7bc84d26&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:00.549256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.549515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:00.550809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:00.575557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:00.576694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:00.577748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:00.577748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:00.577748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:00.577748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:00.583719 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 255/531] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:00.585238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:00.585355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:00.682683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:00.690010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e2759-551c-4c24-9ffd-61ea11faf533 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:00.739661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93af8-fff4-4723-a580-498f0902c10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:00.740604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93af8-fff4-4723-a580-498f0902c10f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:00.741113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93af8-fff4-4723-a580-498f0902c10f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:00.741561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93af8-fff4-4723-a580-498f0902c10f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:00.779742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d9aad-deae-4630-9d49-108eb4a483d7 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:00.838853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b93af8-fff4-4723-a580-498f0902c10f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.840506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b93af8-fff4-4723-a580-498f0902c10f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.869888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:00.901257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:00.901958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:00.902301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:00.902504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:00.903240 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:00.903550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:00.905014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:00.919610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 256/532] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:00.928282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:00.987494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 277/533] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:00.994798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93af8-fff4-4723-a580-498f0902c10f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:01.078164 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc078bfd-ab97-4bdf-b81e-f07a21d8b066 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:01.120964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:01.146165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:01.146967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:01.147250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:01.147525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:01.160619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:01.162096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:01.181165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93af8-fff4-4723-a580-498f0902c10f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:01.194445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:01.195618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:01.196543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 257/534] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:01.203152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:01.215788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:01.224545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:01.224545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:01.224545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:01.228889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:01.230460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:01.257407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:01.258353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:01.259210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:01.259210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:01.259210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:01.259210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:01.259210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:01.260053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 278/535] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:01.265002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:01.352963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1baeea-6d44-4c58-b454-4ce859df25a5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:01.391046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:01.391934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:01.392361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:01.392801 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:01.393981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:01.394810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:01.396395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:01.416652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 258/536] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:01.423394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:01.509789 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a3c2a-dd8a-4996-895a-5d39fffe0eb4 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:01.509991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:01.566221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:01.567349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:01.571384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:01.571875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:01.572951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:01.573478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:01.575115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:01.598626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 279/537] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:01.603159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:01.626180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23732386-73f0-4532-8a6c-61ec6a45e730 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:01.672852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:01.672852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:01.672852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:01.672852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:01.675740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=c958b233374b4013a8bbe9efbed054a2) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:01.676858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:01.680145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:01.689534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 259/538] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:01 2022] GET /identity/v3/role_assignments?user.id=c958b233374b4013a8bbe9efbed054a2&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:01.708691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:01.823271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da9a7-3b67-4747-a6f3-2982e0a43b3c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:01.877919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:01.878743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:01.879029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:01.879324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:01.892461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:01.893962 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:01.910657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:01.911528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:01.912384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:01.912384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:01.912384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:01.912384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:01.912384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:01.912384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:01.912384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:01.912384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:01.915410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 280/539] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:01.922124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:02.042731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcae8d-b162-4c87-9f79-e673b8295049 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:02.054808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:02.092074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722a934-a002-4d96-a45e-fd59f6e19168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:02.092830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a934-a002-4d96-a45e-fd59f6e19168 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:02.093112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a934-a002-4d96-a45e-fd59f6e19168 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:02.093393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a934-a002-4d96-a45e-fd59f6e19168 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:02.094241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722a934-a002-4d96-a45e-fd59f6e19168 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:02.094599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722a934-a002-4d96-a45e-fd59f6e19168 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:02.096035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722a934-a002-4d96-a45e-fd59f6e19168 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:02.112116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 260/540] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:02.118227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:02.237128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:02.241535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee970ca-f339-492f-9857-5dc99987477b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:02.244286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34348ff1-00ac-463e-9d11-bb52a43d37d8 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:02.244726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34348ff1-00ac-463e-9d11-bb52a43d37d8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:02.244990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34348ff1-00ac-463e-9d11-bb52a43d37d8 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:02.246396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 261/541] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:02.251448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08af87-5398-4f6a-9f58-5b597abffcda None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:02.251761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08af87-5398-4f6a-9f58-5b597abffcda None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:02.252013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08af87-5398-4f6a-9f58-5b597abffcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:02.283879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-2e08af87-5398-4f6a-9f58-5b597abffcda None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:02.285385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2e08af87-5398-4f6a-9f58-5b597abffcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:02.307534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:02.308545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:02.309034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:02.310009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:02.311330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=96df1ee2cbc14bcc8a8a250fd8f349dd) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:02.311847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:02.320556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:02.333072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 281/542] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:02 2022] GET /identity/v3/role_assignments?user.id=96df1ee2cbc14bcc8a8a250fd8f349dd&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:02.388612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 262/543] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:02.442010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:02.561764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ae553-fef1-49de-9955-8ce81f1ac913 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:02.605003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038d210-35d4-424a-bce3-37382d574157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:02.605821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038d210-35d4-424a-bce3-37382d574157 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:02.606108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038d210-35d4-424a-bce3-37382d574157 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:02.606391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038d210-35d4-424a-bce3-37382d574157 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:02.627174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038d210-35d4-424a-bce3-37382d574157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-6038d210-35d4-424a-bce3-37382d574157 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:02.635556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:02.636468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:02.637323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:02.640005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 282/544] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:02.645634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08af87-5398-4f6a-9f58-5b597abffcda None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:02.769761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:02.780338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08af87-5398-4f6a-9f58-5b597abffcda None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:02.819413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0dfd3-a3bc-4886-aba9-2152709ed936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:02.820007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0dfd3-a3bc-4886-aba9-2152709ed936 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:02.820288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0dfd3-a3bc-4886-aba9-2152709ed936 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:02.820578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0dfd3-a3bc-4886-aba9-2152709ed936 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:02.821373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0dfd3-a3bc-4886-aba9-2152709ed936 None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:02.821678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0dfd3-a3bc-4886-aba9-2152709ed936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:02.823046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0dfd3-a3bc-4886-aba9-2152709ed936 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:02.835670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 263/545] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:02.841006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038d210-35d4-424a-bce3-37382d574157 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:02.878555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:02.951667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142531f0-6af7-4b63-9b0f-d2edd13e5c26 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:02.951986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142531f0-6af7-4b63-9b0f-d2edd13e5c26 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:02.952234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142531f0-6af7-4b63-9b0f-d2edd13e5c26 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:02.954110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 264/546] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:02.960018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47adb3a-692a-4f11-b18b-7ac726bab0f7 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:02.960018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47adb3a-692a-4f11-b18b-7ac726bab0f7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:02.960018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47adb3a-692a-4f11-b18b-7ac726bab0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:02.969434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038d210-35d4-424a-bce3-37382d574157 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.002633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d47adb3a-692a-4f11-b18b-7ac726bab0f7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:03.003839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d47adb3a-692a-4f11-b18b-7ac726bab0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:03.027490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.028286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.028591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.028872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.045730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:03.048444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:03.066740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:03.067728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:03.068626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:03.068626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:03.068626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:03.068626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:03.068626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 283/547] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:03.077526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.142535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 265/548] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:17:03.158005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:03.186732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47adb3a-692a-4f11-b18b-7ac726bab0f7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.187921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbb0c9-048e-4b7c-8015-eadf64c9ba8f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.223712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a3204-ea0b-4723-990d-99c2144d7913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.224550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a3204-ea0b-4723-990d-99c2144d7913 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.224860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a3204-ea0b-4723-990d-99c2144d7913 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.225140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a3204-ea0b-4723-990d-99c2144d7913 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.225872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a3204-ea0b-4723-990d-99c2144d7913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:03.226181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a3204-ea0b-4723-990d-99c2144d7913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:03.227604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a3204-ea0b-4723-990d-99c2144d7913 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:03.244152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 284/549] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:03.245770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847b326-a7aa-43b9-9124-e3350725a4fd None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.246065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847b326-a7aa-43b9-9124-e3350725a4fd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.246345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847b326-a7aa-43b9-9124-e3350725a4fd None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.247888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 285/550] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:03.249810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847b326-a7aa-43b9-9124-e3350725a4fd None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.326566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47adb3a-692a-4f11-b18b-7ac726bab0f7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.348825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:03.363916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffd07c-b583-40c6-840f-39f6b1aae23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.367142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffd07c-b583-40c6-840f-39f6b1aae23d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.367142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffd07c-b583-40c6-840f-39f6b1aae23d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.367142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffd07c-b583-40c6-840f-39f6b1aae23d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.373811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847b326-a7aa-43b9-9124-e3350725a4fd None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.378069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffd07c-b583-40c6-840f-39f6b1aae23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-fdffd07c-b583-40c6-840f-39f6b1aae23d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:03.387842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:03.389005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:03.390028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:03.396646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 266/551] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:03.398286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed63b4-dd38-4818-9211-313f02c19d00 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.398716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed63b4-dd38-4818-9211-313f02c19d00 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.399168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed63b4-dd38-4818-9211-313f02c19d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.408732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.409530 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.409815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.410094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] PATH_INFO: `/v3/users/swift` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.412223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:03.418625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:03.422738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:03.422938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-44ed63b4-dd38-4818-9211-313f02c19d00 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:03.422938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-44ed63b4-dd38-4818-9211-313f02c19d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:03.435702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:03.437007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:03.437928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 286/552] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:03.438521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.499413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 267/553] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:03.500610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed63b4-dd38-4818-9211-313f02c19d00 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.542104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3457bd-35fa-4a45-8642-9a21fcfeb61a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.581016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.581693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.581978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.582255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.583011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:03.583321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:03.585425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:03.596245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:03.607521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 287/554] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:03.607521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.631348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed63b4-dd38-4818-9211-313f02c19d00 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.677209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:03.679731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.681274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.681565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.681842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.683542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:03.683542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:03.684818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:03.708452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 268/555] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:03.709689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.736258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38531b6c-a6b0-4baa-9933-99a4e382e030 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.773011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0020bb1-0fdf-4167-9b44-393d406c69de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.774622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0020bb1-0fdf-4167-9b44-393d406c69de None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.775115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0020bb1-0fdf-4167-9b44-393d406c69de None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.775537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0020bb1-0fdf-4167-9b44-393d406c69de None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.794870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0020bb1-0fdf-4167-9b44-393d406c69de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-c0020bb1-0fdf-4167-9b44-393d406c69de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:03.806182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:03.807161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:03.807994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:03.809167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 288/556] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:03.812284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0020bb1-0fdf-4167-9b44-393d406c69de None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.838039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e0510-abd3-423d-8369-82f0e1407fa5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.884027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.885117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.885654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.885940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:03.889781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:03.899140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:03.900966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:03.923888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:03.926275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:03.927985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:03.927985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:03.927985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:03.927985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:03.927985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 269/557] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:03.927985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:03.944754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0020bb1-0fdf-4167-9b44-393d406c69de None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:03.967070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:03.990115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0ccdd-7851-44b6-a325-9bc3841f86e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:03.990817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ccdd-7851-44b6-a325-9bc3841f86e9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:03.991111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ccdd-7851-44b6-a325-9bc3841f86e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:03.991393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ccdd-7851-44b6-a325-9bc3841f86e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.043151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebcc3f-a8da-477b-9676-7d325b02332e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.080585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0ccdd-7851-44b6-a325-9bc3841f86e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.082156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0ccdd-7851-44b6-a325-9bc3841f86e9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.091669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:04.092354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.093105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.093408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.094171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:04.094981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.096652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.108755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 270/558] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:04.109754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.174483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:04.200479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 289/559] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:04.206027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0ccdd-7851-44b6-a325-9bc3841f86e9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.224797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:04.229796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948ddb5-2dc0-4ba7-8781-f5e1a63ecfec None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.269110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:04.270048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.270339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.270619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.271402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:04.271817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.273435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.287112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 271/560] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:04.288080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.312225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0ccdd-7851-44b6-a325-9bc3841f86e9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.346615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:04.347415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.347696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.347973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.359178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.360760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:04.380371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:04.382498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:04.384780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:04.384780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:04.384780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:04.384780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:04.384780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 290/561] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:04.384780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.403136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e357a-6817-4315-9502-8742b7ea159b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.415868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:04.442481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:04.443346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.443634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.443913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.454926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.456378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.458636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:04.476168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:04.479404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:04.482657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:04.482657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:04.482657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:04.482657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:04.482657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 272/562] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:04.483536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.494056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855984cb-ede8-4431-b175-f78db92d48d6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.534882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d158564-867e-4704-9a7e-6b40106f81c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:04.535684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d158564-867e-4704-9a7e-6b40106f81c7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.535971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d158564-867e-4704-9a7e-6b40106f81c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.536253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d158564-867e-4704-9a7e-6b40106f81c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.544570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d158564-867e-4704-9a7e-6b40106f81c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.546003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d158564-867e-4704-9a7e-6b40106f81c7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-4d158564-867e-4704-9a7e-6b40106f81c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:04.560371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:04.561388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:04.562352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:04.566237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 291/563] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:04.567414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1db206-41b6-475e-afd0-115e37328252 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.567709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1db206-41b6-475e-afd0-115e37328252 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.567991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1db206-41b6-475e-afd0-115e37328252 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.569406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 292/564] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:04.574838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1db206-41b6-475e-afd0-115e37328252 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.595949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739081e-5a1f-41df-b821-62e137621233 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.634963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:04.635644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.635929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.636211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.636999 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:04.637309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.638742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.652571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 273/565] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:04.653375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.681459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:04.700573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1db206-41b6-475e-afd0-115e37328252 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.772963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:04.780803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b503818-b4ad-4f44-97c3-56b1aa1bd48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:04.781468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b503818-b4ad-4f44-97c3-56b1aa1bd48c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.781749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b503818-b4ad-4f44-97c3-56b1aa1bd48c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.782027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b503818-b4ad-4f44-97c3-56b1aa1bd48c None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.782748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b503818-b4ad-4f44-97c3-56b1aa1bd48c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:04.783063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b503818-b4ad-4f44-97c3-56b1aa1bd48c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.784422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b503818-b4ad-4f44-97c3-56b1aa1bd48c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.795680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 293/566] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:04.796914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f4e59-c84a-451a-9c3e-a90d44ca78f7 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.797207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f4e59-c84a-451a-9c3e-a90d44ca78f7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.797488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f4e59-c84a-451a-9c3e-a90d44ca78f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.814196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d16f4e59-c84a-451a-9c3e-a90d44ca78f7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:04.815371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d16f4e59-c84a-451a-9c3e-a90d44ca78f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:04.815875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509e34-5563-485d-a66e-8272c7e0590b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.849594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:04.850267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.850550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.850836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.851665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:04.852483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:04.853949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:04.870784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 274/567] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:04.871570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.881893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 294/568] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:04.883802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac635e5-7d81-49a0-bebb-6b57077c412d None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:04.884094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac635e5-7d81-49a0-bebb-6b57077c412d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:04.884373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac635e5-7d81-49a0-bebb-6b57077c412d None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:04.885936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 295/569] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:04.886609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac635e5-7d81-49a0-bebb-6b57077c412d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:04.952855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:04.971046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275368b1-d483-4fae-a703-e8125ff3addc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:04.995009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac635e5-7d81-49a0-bebb-6b57077c412d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.004629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.005420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.005709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.005984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.011474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.013152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.014575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.027195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc89cd0-be80-4675-8b01-20be8b5bafef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.027978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89cd0-be80-4675-8b01-20be8b5bafef None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.028262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89cd0-be80-4675-8b01-20be8b5bafef None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.028561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89cd0-be80-4675-8b01-20be8b5bafef None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:05.029029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:05.030031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:05.030989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 275/570] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:05.031579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.040658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc89cd0-be80-4675-8b01-20be8b5bafef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.042217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc89cd0-be80-4675-8b01-20be8b5bafef None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-9fc89cd0-be80-4675-8b01-20be8b5bafef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:05.057253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:05.058149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:05.059464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 296/571] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:05.060081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd56e-2ae0-4066-a416-5e4010791849 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.060081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd56e-2ae0-4066-a416-5e4010791849 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.060359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd56e-2ae0-4066-a416-5e4010791849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.076185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e03fd56e-2ae0-4066-a416-5e4010791849 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:05.077387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e03fd56e-2ae0-4066-a416-5e4010791849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:05.130430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926351df-3017-416d-9d2a-65a24b4bf27c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.141257 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 297/572] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:05.142072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fd56e-2ae0-4066-a416-5e4010791849 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.168057 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.168869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.169159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.169437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.177793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.179860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.181959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:05.198834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:05.199750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:05.200966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:05.200966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:05.200966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:05.200966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:05.200966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 276/573] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:05.200966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.249005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fd56e-2ae0-4066-a416-5e4010791849 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.258450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.283403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca1b28-3db9-4bc3-9edb-320135ddce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.284194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca1b28-3db9-4bc3-9edb-320135ddce7d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.284482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca1b28-3db9-4bc3-9edb-320135ddce7d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.284797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca1b28-3db9-4bc3-9edb-320135ddce7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.295435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca1b28-3db9-4bc3-9edb-320135ddce7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:05.301053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415adf-9614-482a-a23c-37997b777db9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-90ca1b28-3db9-4bc3-9edb-320135ddce7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:05.303277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:05.304369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 298/574] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:05.305446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca1b28-3db9-4bc3-9edb-320135ddce7d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.335354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.336020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.336832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.337115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.337940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:05.338246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.339654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.356741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 277/575] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:05.357338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.405289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.409007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca1b28-3db9-4bc3-9edb-320135ddce7d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.442609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.443263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.443543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.443820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.444632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:05.444943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.446387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.454224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae55be-3966-4dc3-b4ff-fa82ac097b6c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.458031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 299/576] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:05.458817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.486450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.487958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46ef3-0848-41f4-be85-be2ea6f2bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.488771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46ef3-0848-41f4-be85-be2ea6f2bccc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.489053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46ef3-0848-41f4-be85-be2ea6f2bccc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.489330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46ef3-0848-41f4-be85-be2ea6f2bccc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.499269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b46ef3-0848-41f4-be85-be2ea6f2bccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-65b46ef3-0848-41f4-be85-be2ea6f2bccc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:05.507993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:05.509088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:05.510011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 278/577] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:05.510709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46ef3-0848-41f4-be85-be2ea6f2bccc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.558884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f9bc7-3258-4f75-b767-3f2e4725f01d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.593867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.594533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.594812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.595084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.595796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:05.596104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.597476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.608968 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46ef3-0848-41f4-be85-be2ea6f2bccc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.609434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 300/578] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:05.610134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.622954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.645434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.646102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.646392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.646690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.647435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:05.647996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.649662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.657961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 279/579] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:05.658513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.702154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.714698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a20c59-d7cf-4fba-8902-09e5fde14015 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.756922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.757710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.757993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.758269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.772527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.774122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.788796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dd9f6-60dd-4cdb-a226-15b82038faef None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:05.791109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:05.792009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:05.793031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:05.793031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:05.793031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:05.793031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:05.793031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 301/580] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:05.793313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.824916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.825565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.825851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.826131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.826961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=645bbe41756749b98232d0e2eb2e678a) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:05.827280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.828715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.834091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 280/581] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/role_assignments?user.id=645bbe41756749b98232d0e2eb2e678a&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:05.834944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.875234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.905988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde1426-511c-439f-8eeb-f82ee57b1b05 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.944369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.945122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.945412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.945729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:05.946484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:05.946791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:05.948169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:05.952022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085341c-683c-4c9e-997c-862f4f0e68ff None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:05.957167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 302/582] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:05.958031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:05.964981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:05.993424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:05.994229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:05.994517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:05.994790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.003116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.004799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:06.018824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:06.019909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:06.021100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:06.021100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:06.021100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:06.021100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:06.021100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:06.021100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:06.021100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:06.021100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:06.022922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 281/583] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:06.024366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.074365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43863837-f4ec-4c15-ba30-2a37f5234dc7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.109527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.110320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.110605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.110881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.121606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.123036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.123266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:06.129639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae225a-60ac-4665-825a-510f157dc450 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:06.140053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:06.141049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:06.141956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:06.141956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:06.141956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:06.141956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:06.141956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 303/584] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:06.142668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.170113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.170898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.171179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.171456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.172184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:06.172483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.174360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.190541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 282/585] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:06.190541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.204503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:06.255272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a70701-f76b-40e3-b3e0-c4ddbcc5ca55 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.293963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.294167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.294414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.294517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a416-c253-4ea6-9702-78bb3830a01e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.294603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.307951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.309449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:06.327388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:06.328468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:06.329439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:06.329439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:06.329439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:06.329439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:06.329439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 304/586] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:06.329723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.335429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.337520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.337815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.338091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.338885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:06.339193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.340650 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.354297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 283/587] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:06.354925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.374323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:06.442402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb95f6-e744-45c4-9cba-173e7746ef86 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.451525 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:06.462841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88493-fde0-4c58-8c9d-1f366a2d1ca1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.484107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.484874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.485158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.485434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.486156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:06.486458 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.487818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.500786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 305/588] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:06.501034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.501825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.502108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.502384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.505886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.509354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.510778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:06.525483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:06.526445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:06.527442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 284/589] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:06.527937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.614500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:06.619640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdf622-590a-4d6f-8ebc-0a9b0d8026c9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.638719 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637803b-3348-44c9-acda-f859a61d43cc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.653181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.653842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.654125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.654400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.655113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:06.655418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.656794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.674049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 306/590] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:06.674845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.680102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71275c78-f599-4435-aa88-27361f7dd290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.681532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71275c78-f599-4435-aa88-27361f7dd290 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.681815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71275c78-f599-4435-aa88-27361f7dd290 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.682091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71275c78-f599-4435-aa88-27361f7dd290 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/dea47e0ec737422c8f679e21174f8cab/roles/78418cd7aaaf481e94926d572280f81e` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.697114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71275c78-f599-4435-aa88-27361f7dd290 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=dea47e0ec737422c8f679e21174f8cab, role_id=78418cd7aaaf481e94926d572280f81e)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.702397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71275c78-f599-4435-aa88-27361f7dd290 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.703520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:06.782402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24829102-bc4e-4510-b060-7294e45abfb9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.796833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 285/591] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:06 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/dea47e0ec737422c8f679e21174f8cab/roles/78418cd7aaaf481e94926d572280f81e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:06.798383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71275c78-f599-4435-aa88-27361f7dd290 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.819878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:06.820761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:06.821050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:06.821385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:06.829371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:06.830845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:06.847186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:06.848216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:06.849256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 307/592] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:06.849868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:06.964772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71275c78-f599-4435-aa88-27361f7dd290 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.976556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:06.998294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffb0d3-f638-4ae9-bddf-b0ed38654768 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:06.998964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:07.007973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.008787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.009073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.009350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.010275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:07.018698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.018698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.025034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 286/593] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:07.025936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:07.034598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.035463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.035754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.036031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.044835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.046343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:07.063334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:07.068020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 308/594] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:07.070311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:07.132988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d99d5-53bf-4b38-91ae-5e559a7ed66e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:07.168172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.168983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.169272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.169549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.170386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:07.170704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.172347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.175003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1abf3f-cc29-493a-b3af-33e942c9b0e3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:07.192208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 287/595] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jun 16 00:17:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:07.197510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:07.210112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc684d-c588-4b4a-b1fd-145da164dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.211055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc684d-c588-4b4a-b1fd-145da164dcdd None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.211947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc684d-c588-4b4a-b1fd-145da164dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.212215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc684d-c588-4b4a-b1fd-145da164dcdd None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/fff82e4b1a4248fcbb3566bb7bc84d26/roles/78418cd7aaaf481e94926d572280f81e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.228326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc684d-c588-4b4a-b1fd-145da164dcdd None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=fff82e4b1a4248fcbb3566bb7bc84d26, role_id=78418cd7aaaf481e94926d572280f81e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.229883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc684d-c588-4b4a-b1fd-145da164dcdd None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.232954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:07.247066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:07.301584 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c0d5-b206-4ea5-a7d6-68427f6f0521 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:07.306006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 309/596] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:07 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/fff82e4b1a4248fcbb3566bb7bc84d26/roles/78418cd7aaaf481e94926d572280f81e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:07.314469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc684d-c588-4b4a-b1fd-145da164dcdd None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:07.346704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.348697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.348697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.348697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.349184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:07.349184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.350696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.374899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 288/597] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Jun 16 00:17:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:07.381310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:07.502367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445b9a8-f87e-455d-8ed5-afc9dd88b4d3 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:07.504861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:07.545463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.546771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.547270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.547876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.559806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.561827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:07.581538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:07.582501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:07.583424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:07.583424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:07.583424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:07.583424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:07.585928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 289/598] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:07.594015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:07.607658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc684d-c588-4b4a-b1fd-145da164dcdd None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:07.662507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.663564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.663856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.664140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.677955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.679437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:07.699477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:07.700421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:07.701361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:07.701361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:07.701361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:07.701361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:07.701361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 310/599] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:07.707683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:07.770613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cdfef-379a-4562-8932-e033c62ee566 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:07.812118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:07.821793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.822464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.822752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.823032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.823758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:07.824069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.825470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.838301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 290/600] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:07.843653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:07.871940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:07.920218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993429-6750-43b0-a444-ba3078d9f2a2 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:07.961201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:07.962833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:07.963426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:07.963898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:07.964861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:07.965367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:07.967040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:07.980231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 311/601] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:07.985948 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:08.014382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fad1cc-88df-430b-bb08-74b1a5c15a7f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:08.077004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c2e06-8acc-4e8f-b291-a8f4e55f113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:08.078179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c2e06-8acc-4e8f-b291-a8f4e55f113c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.078715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c2e06-8acc-4e8f-b291-a8f4e55f113c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.078825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c2e06-8acc-4e8f-b291-a8f4e55f113c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.086158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c2e06-8acc-4e8f-b291-a8f4e55f113c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:08.087655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c2e06-8acc-4e8f-b291-a8f4e55f113c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:08.094096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1a87b-3678-4d3e-9613-50132823b197 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-5d8c2e06-8acc-4e8f-b291-a8f4e55f113c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:08.103079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:08.104027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:08.104986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:08.104986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:08.104986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:08.104986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:08.104986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:08.104986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:08.104986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:08.104986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:08.108952 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 291/602] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:08.109517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:08.111085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044f85-5133-4ca6-b7a9-23cddb70d321 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.111509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044f85-5133-4ca6-b7a9-23cddb70d321 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.111792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044f85-5133-4ca6-b7a9-23cddb70d321 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.113352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 292/603] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:08.114056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044f85-5133-4ca6-b7a9-23cddb70d321 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:08.137181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9a4ea-7bb0-48a6-a066-fa093e665bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:08.138342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a4ea-7bb0-48a6-a066-fa093e665bb8 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.138951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a4ea-7bb0-48a6-a066-fa093e665bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.139428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a4ea-7bb0-48a6-a066-fa093e665bb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.146539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9a4ea-7bb0-48a6-a066-fa093e665bb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:08.148202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9a4ea-7bb0-48a6-a066-fa093e665bb8 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-cec9a4ea-7bb0-48a6-a066-fa093e665bb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:08.162835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:08.163812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:08.165294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:08.165294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:08.165294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:08.165294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:08.165294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:08.165294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:08.165294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:08.165294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:08.167736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 312/604] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:08.174071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebbe5-c88b-4709-ac43-0cd8e3e1c42a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.174614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebbe5-c88b-4709-ac43-0cd8e3e1c42a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.175104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebbe5-c88b-4709-ac43-0cd8e3e1c42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.192576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-144ebbe5-c88b-4709-ac43-0cd8e3e1c42a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:08.194146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-144ebbe5-c88b-4709-ac43-0cd8e3e1c42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:08.204376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:08.233835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044f85-5133-4ca6-b7a9-23cddb70d321 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:08.264804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 313/605] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:08.266057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ebbe5-c88b-4709-ac43-0cd8e3e1c42a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:08.281866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:08.282539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.282872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.283097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.283953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:08.284265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:08.286114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:08.306503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 293/606] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:08.313576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:08.365805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:08.369921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ebbe5-c88b-4709-ac43-0cd8e3e1c42a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:08.405470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:08.406454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.406952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.407445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.408475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:08.409008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:08.410612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:08.423391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 314/607] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:08.428933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:08.438550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef7f9-8547-4a1d-97c7-dd3f0457c1a5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:08.475642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044ab6-f930-4773-8f16-01e5910f4610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:08.476563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044ab6-f930-4773-8f16-01e5910f4610 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.477489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044ab6-f930-4773-8f16-01e5910f4610 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.477951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044ab6-f930-4773-8f16-01e5910f4610 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/96df1ee2cbc14bcc8a8a250fd8f349dd/roles/78418cd7aaaf481e94926d572280f81e` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.480746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:08.495038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3044ab6-f930-4773-8f16-01e5910f4610 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=96df1ee2cbc14bcc8a8a250fd8f349dd, role_id=78418cd7aaaf481e94926d572280f81e)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:08.496861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3044ab6-f930-4773-8f16-01e5910f4610 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:08.531343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e8feb-320d-4c4b-ab13-a112fdc5536a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:08.567220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5b62b-8ff2-40e7-8bc0-c9e251ab6bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:08.568005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5b62b-8ff2-40e7-8bc0-c9e251ab6bb1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.568287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5b62b-8ff2-40e7-8bc0-c9e251ab6bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.568583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5b62b-8ff2-40e7-8bc0-c9e251ab6bb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.582325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5b62b-8ff2-40e7-8bc0-c9e251ab6bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-44a5b62b-8ff2-40e7-8bc0-c9e251ab6bb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:08.597124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:08.598163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 315/608] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:08.599022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5b62b-8ff2-40e7-8bc0-c9e251ab6bb1 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:08.624644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 294/609] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:08 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/96df1ee2cbc14bcc8a8a250fd8f349dd/roles/78418cd7aaaf481e94926d572280f81e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:08.626187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044ab6-f930-4773-8f16-01e5910f4610 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:08.636623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:08.817560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5b62b-8ff2-40e7-8bc0-c9e251ab6bb1 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:08.835560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044ab6-f930-4773-8f16-01e5910f4610 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:08.863564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799faa1b-6024-449d-93e1-9e922fab6280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:08.864797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799faa1b-6024-449d-93e1-9e922fab6280 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.865093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799faa1b-6024-449d-93e1-9e922fab6280 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.865416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799faa1b-6024-449d-93e1-9e922fab6280 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/c958b233374b4013a8bbe9efbed054a2/roles/78418cd7aaaf481e94926d572280f81e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.876610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:08.877562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:08.878213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:08.878790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:08.879847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:08.880379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:08.882264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799faa1b-6024-449d-93e1-9e922fab6280 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=c958b233374b4013a8bbe9efbed054a2, role_id=78418cd7aaaf481e94926d572280f81e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:08.882604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:08.884262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799faa1b-6024-449d-93e1-9e922fab6280 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:08.893608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 295/610] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:08.899448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:08.951684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:08.962066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 316/611] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:08 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/c958b233374b4013a8bbe9efbed054a2/roles/78418cd7aaaf481e94926d572280f81e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:09.043690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd02744-cb61-4e4c-85f8-ccf007381cd0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:09.103496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a32fa-d020-4c36-9df8-cedeaa4d1835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:09.104310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a32fa-d020-4c36-9df8-cedeaa4d1835 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.104621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a32fa-d020-4c36-9df8-cedeaa4d1835 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.104904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a32fa-d020-4c36-9df8-cedeaa4d1835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.127059 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a32fa-d020-4c36-9df8-cedeaa4d1835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:09.128444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a32fa-d020-4c36-9df8-cedeaa4d1835 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-144a32fa-d020-4c36-9df8-cedeaa4d1835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:09.152539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:09.153515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:09.154406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:09.154406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:09.154406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:09.154406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:09.156043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 296/612] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:09.161988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799faa1b-6024-449d-93e1-9e922fab6280 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:09.244624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:09.286551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799faa1b-6024-449d-93e1-9e922fab6280 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:09.341241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf09d2-2cc9-4264-ab60-cca56c84c9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:09.341933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf09d2-2cc9-4264-ab60-cca56c84c9aa None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.342223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf09d2-2cc9-4264-ab60-cca56c84c9aa None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.342501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf09d2-2cc9-4264-ab60-cca56c84c9aa None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.343233 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf09d2-2cc9-4264-ab60-cca56c84c9aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:09.343541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf09d2-2cc9-4264-ab60-cca56c84c9aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:09.344944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf09d2-2cc9-4264-ab60-cca56c84c9aa None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:09.355880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becdc46-4fbc-4462-8cf8-0ab9c0899bd7 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.356473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becdc46-4fbc-4462-8cf8-0ab9c0899bd7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.356989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becdc46-4fbc-4462-8cf8-0ab9c0899bd7 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.358882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 297/613] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:09.362416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 317/614] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:09.368237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becdc46-4fbc-4462-8cf8-0ab9c0899bd7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:09.374982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4565-64d5-4363-a531-c7fa24b35c60 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.375275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4565-64d5-4363-a531-c7fa24b35c60 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.375562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4565-64d5-4363-a531-c7fa24b35c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.410074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5d6c4565-64d5-4363-a531-c7fa24b35c60 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:09.411184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5d6c4565-64d5-4363-a531-c7fa24b35c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:09.475018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:09.481326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becdc46-4fbc-4462-8cf8-0ab9c0899bd7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:09.519534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:09.520805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.521307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.521778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] PATH_INFO: `/v3/users/swift` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.530676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:09.532683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:09.548574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:09.549597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:09.550561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:09.553357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 298/615] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:09.559597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:09.566178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 318/616] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:17:09.602578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c4565-64d5-4363-a531-c7fa24b35c60 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:09.674891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81552c63-60fe-458e-bb17-5cb7d1224b2e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:09.710973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456cb88-bbf1-4298-a7b7-5988a6ca12a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:09.712289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456cb88-bbf1-4298-a7b7-5988a6ca12a5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.713655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456cb88-bbf1-4298-a7b7-5988a6ca12a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.713655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456cb88-bbf1-4298-a7b7-5988a6ca12a5 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.713834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456cb88-bbf1-4298-a7b7-5988a6ca12a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:09.714081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456cb88-bbf1-4298-a7b7-5988a6ca12a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:09.715503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456cb88-bbf1-4298-a7b7-5988a6ca12a5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:09.735090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 299/617] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jun 16 00:17:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:09.736430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3ff2-1abc-4067-9afd-4955128a975b None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.741445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3ff2-1abc-4067-9afd-4955128a975b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.741760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3ff2-1abc-4067-9afd-4955128a975b None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.743404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 300/618] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:09.744278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e3ff2-1abc-4067-9afd-4955128a975b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:09.747305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:09.764159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:09.798396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c4565-64d5-4363-a531-c7fa24b35c60 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:09.847549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e608a-04bc-426a-be50-72c7fb08a574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:09.848644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e608a-04bc-426a-be50-72c7fb08a574 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.848930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e608a-04bc-426a-be50-72c7fb08a574 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.849205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e608a-04bc-426a-be50-72c7fb08a574 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.854523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e3ff2-1abc-4067-9afd-4955128a975b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:09.863008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e608a-04bc-426a-be50-72c7fb08a574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-724e608a-04bc-426a-be50-72c7fb08a574 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:09.873526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:09.874641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:09.875733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 319/619] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:09.876710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff6f8-146d-468b-9e03-8a3d540f3f1e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.880544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff6f8-146d-468b-9e03-8a3d540f3f1e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.880544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff6f8-146d-468b-9e03-8a3d540f3f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.894194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f66ff6f8-146d-468b-9e03-8a3d540f3f1e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:09.895453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f66ff6f8-146d-468b-9e03-8a3d540f3f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:09.900229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:09.901099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:09.901386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:09.901663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:09.913053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:09.915301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:09.935072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:09.936019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:09.936956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:09.936956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:09.936956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:09.936956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:09.936956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 301/620] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:09.941547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:09.968642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 320/621] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:09.969722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ff6f8-146d-468b-9e03-8a3d540f3f1e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.015680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:10.020916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:10.053888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb9839-c5e0-4bd2-9c4b-47dfb84c4612 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.075866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ff6f8-146d-468b-9e03-8a3d540f3f1e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.091705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.092828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.093164 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.093443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.094280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:10.094563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.095968 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.108867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 302/622] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:10.108867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.118292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.119169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.119609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.120068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.120974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:10.121436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.122962 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.139867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 321/623] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:10.140987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.216638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f79a0-9426-4ad3-81af-e2051b083c5d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.244405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a3658-75ac-4bec-8370-7db38b5273c1 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.247509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:10.253882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:10.254409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279732c4-b0bd-4233-8ca3-78fe8643ca3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.255203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279732c4-b0bd-4233-8ca3-78fe8643ca3e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.255541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279732c4-b0bd-4233-8ca3-78fe8643ca3e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.256031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279732c4-b0bd-4233-8ca3-78fe8643ca3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.266847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279732c4-b0bd-4233-8ca3-78fe8643ca3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-279732c4-b0bd-4233-8ca3-78fe8643ca3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:10.274890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:10.276030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:10.277062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 303/624] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:10.277818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279732c4-b0bd-4233-8ca3-78fe8643ca3e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.283274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.284252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.284745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.285190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.297000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.298870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:10.316676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:10.317788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:10.318842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:10.318842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:10.318842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:10.318842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:10.324790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 322/625] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:10.325778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.388229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279732c4-b0bd-4233-8ca3-78fe8643ca3e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.427237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.428128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.428415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.429009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.432511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8594a6c-b816-4032-a7bd-fc4bde75ef3f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.436761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.438193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:10.455218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:10.456402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:10.457512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:10.457512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:10.457512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:10.457512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:10.457512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:10.457512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:10.457512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:10.457512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:10.461697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 304/626] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:10.465962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.476441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.478209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.478650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.479078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.479993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:10.480472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.482132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.491471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 323/627] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:10.492555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.516817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:10.519674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:10.571926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a55f1-bf2b-4100-9e79-9bc6b36cc20d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.598721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc89b9-3029-4e6e-8f86-2c962b825935 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.610042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.610949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.611375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.611822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.612729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:10.613179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.614683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.627325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 305/628] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:10.628346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.639195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.640105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.640105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.640403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.641230 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:10.641548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.643040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.655687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 324/629] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:10.656510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.738443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023bc5b-c7f0-492a-940c-660061c0b64e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.743134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:10.751332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:10.762063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f1d22-a86b-4dbd-a359-5d207fb91587 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.776913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.778084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.778606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.779090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.791394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.793293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.798056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.799882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.799882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.799882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.807979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.809469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:10.815624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:10.816505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:10.817435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:10.817435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:10.817435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:10.817435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:10.822373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 306/630] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:10.823471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:10.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:10.825576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:10.826573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:10.830431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 325/631] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:10.831347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.940821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1135dd-c91e-418a-ae65-f197b1df1e94 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.941469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66e1e-2b93-408f-a25a-cbd627efc5f2 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:10.977432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.978292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.978586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.978872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.979602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:10.979915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:10.980491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d1a9f-5463-420d-be4c-e0c6cbf46090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:10.981311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:10.981860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d1a9f-5463-420d-be4c-e0c6cbf46090 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:10.982353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d1a9f-5463-420d-be4c-e0c6cbf46090 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:10.982827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d1a9f-5463-420d-be4c-e0c6cbf46090 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/645bbe41756749b98232d0e2eb2e678a/roles/78418cd7aaaf481e94926d572280f81e` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:10.995162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 326/632] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:10.995951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:10.998620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:11.000808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d1a9f-5463-420d-be4c-e0c6cbf46090 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=645bbe41756749b98232d0e2eb2e678a, role_id=78418cd7aaaf481e94926d572280f81e)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:11.002631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d1a9f-5463-420d-be4c-e0c6cbf46090 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:11.100629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 307/633] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:10 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/645bbe41756749b98232d0e2eb2e678a/roles/78418cd7aaaf481e94926d572280f81e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:11.102371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d1a9f-5463-420d-be4c-e0c6cbf46090 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:11.104551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9313e-9341-4083-8291-10c394d9eebb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:11.124329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:11.144336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52310f-b877-4ae1-a0d8-bb72cb4d40ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:11.144961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52310f-b877-4ae1-a0d8-bb72cb4d40ef None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.145252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52310f-b877-4ae1-a0d8-bb72cb4d40ef None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.145538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52310f-b877-4ae1-a0d8-bb72cb4d40ef None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.146482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52310f-b877-4ae1-a0d8-bb72cb4d40ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:11.146736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52310f-b877-4ae1-a0d8-bb72cb4d40ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:11.148292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52310f-b877-4ae1-a0d8-bb72cb4d40ef None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:11.174258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 327/634] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:17:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:11.175925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cd9e8-4a1e-4b8e-91de-8c75776e77dc None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.180639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cd9e8-4a1e-4b8e-91de-8c75776e77dc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.180639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cd9e8-4a1e-4b8e-91de-8c75776e77dc None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.180639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 328/635] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:11.180639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cd9e8-4a1e-4b8e-91de-8c75776e77dc None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:11.249494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:11.259874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d1a9f-5463-420d-be4c-e0c6cbf46090 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:11.295407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cd9e8-4a1e-4b8e-91de-8c75776e77dc None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:11.308296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754feec-7ed2-4673-b09f-ab2c5647971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:11.309513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754feec-7ed2-4673-b09f-ab2c5647971e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.310004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754feec-7ed2-4673-b09f-ab2c5647971e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.310506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754feec-7ed2-4673-b09f-ab2c5647971e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.319662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754feec-7ed2-4673-b09f-ab2c5647971e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:11.323249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754feec-7ed2-4673-b09f-ab2c5647971e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:11.333619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfe8c7-6bfe-47a9-9ed6-4482d398cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:11.334282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfe8c7-6bfe-47a9-9ed6-4482d398cb3f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.334571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfe8c7-6bfe-47a9-9ed6-4482d398cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.334855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfe8c7-6bfe-47a9-9ed6-4482d398cb3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-5754feec-7ed2-4673-b09f-ab2c5647971e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:11.344079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:11.345120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:17:11.346089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:11.348692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 308/636] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:11.350131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3862-d326-441c-bb06-5d14a10142a1 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.350427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3862-d326-441c-bb06-5d14a10142a1 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.350704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3862-d326-441c-bb06-5d14a10142a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.374902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8ebe3862-d326-441c-bb06-5d14a10142a1 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:11.376636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8ebe3862-d326-441c-bb06-5d14a10142a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:11.414765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfe8c7-6bfe-47a9-9ed6-4482d398cb3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:11.416229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfe8c7-6bfe-47a9-9ed6-4482d398cb3f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:11.461844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 309/637] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:11.462721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe3862-d326-441c-bb06-5d14a10142a1 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:11.475526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:11.510270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:11.578669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 329/638] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:11.580035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfe8c7-6bfe-47a9-9ed6-4482d398cb3f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:11.604783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe3862-d326-441c-bb06-5d14a10142a1 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:11.645785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:11.646475 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.646770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.647012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.647861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:11.648172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:11.649732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:11.670040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 310/639] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jun 16 00:17:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:11.671191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:11.704170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfe8c7-6bfe-47a9-9ed6-4482d398cb3f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:11.738293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:11.742927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3041996-b5a0-4585-848b-554802476cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:11.743741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3041996-b5a0-4585-848b-554802476cde None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.744025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3041996-b5a0-4585-848b-554802476cde None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.744306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3041996-b5a0-4585-848b-554802476cde None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.755410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3041996-b5a0-4585-848b-554802476cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-d3041996-b5a0-4585-848b-554802476cde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:11.763326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:11.764896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:11.765954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:11.770611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 330/640] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:11.771777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2417b-a381-4991-a13e-c4e37b601794 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.772073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2417b-a381-4991-a13e-c4e37b601794 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.772359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2417b-a381-4991-a13e-c4e37b601794 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.776553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 331/641] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:11.776553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2417b-a381-4991-a13e-c4e37b601794 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:11.788918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7db7-41cb-4e6a-9f3a-eccd12f2c425 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:11.799919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:11.827614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:11.828452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.828764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.829057 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.839300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:11.840758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:11.859590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:11.860620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:11.861558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:11.861558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:11.861558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:11.861558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:11.868092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 311/642] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:11.868092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:11.895616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2417b-a381-4991-a13e-c4e37b601794 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:11.932265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea26b14-c2f4-41c6-b7d1-dfdd71e2f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:11.932909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26b14-c2f4-41c6-b7d1-dfdd71e2f240 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:11.933193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26b14-c2f4-41c6-b7d1-dfdd71e2f240 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:11.933469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26b14-c2f4-41c6-b7d1-dfdd71e2f240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:11.971197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085630a-423b-4ff9-920e-4cdcdfbd71bd None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:11.985665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.005777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7275beb-41d3-43fd-9d0d-a0dddb6a91a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.006630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7275beb-41d3-43fd-9d0d-a0dddb6a91a5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.006885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7275beb-41d3-43fd-9d0d-a0dddb6a91a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.007114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7275beb-41d3-43fd-9d0d-a0dddb6a91a5 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.007739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea26b14-c2f4-41c6-b7d1-dfdd71e2f240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.007882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7275beb-41d3-43fd-9d0d-a0dddb6a91a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:12.008182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7275beb-41d3-43fd-9d0d-a0dddb6a91a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.009467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea26b14-c2f4-41c6-b7d1-dfdd71e2f240 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.009606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7275beb-41d3-43fd-9d0d-a0dddb6a91a5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.020948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 312/643] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:12.023075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874979-4242-4325-973c-7748c8239c36 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.023453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874979-4242-4325-973c-7748c8239c36 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.024277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874979-4242-4325-973c-7748c8239c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.032310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.041743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f8874979-4242-4325-973c-7748c8239c36 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:12.042787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f8874979-4242-4325-973c-7748c8239c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:12.118310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 332/644] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:12.119524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea26b14-c2f4-41c6-b7d1-dfdd71e2f240 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.121042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 313/645] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:12.121930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8874979-4242-4325-973c-7748c8239c36 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.227999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea26b14-c2f4-41c6-b7d1-dfdd71e2f240 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.231792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8874979-4242-4325-973c-7748c8239c36 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.235420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.248858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.263053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.263713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.264001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.264274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.265022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:12.265338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.266684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.269502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.270501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.273676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.273676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.278000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 333/646] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:12.278785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.286591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.288314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:12.309089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:12.310086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:12.311179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:12.311179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:12.311179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:12.311179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:12.317352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 314/647] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:12.318394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.383741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb63134-23f5-40ee-a800-20b75a161fa6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.419322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ceb4d-7d5e-44e2-8180-0709df569c79 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.421548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.422419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.422710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.422989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.434304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.436384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.455061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95404912-22b3-4f26-ace8-79ee76fb30d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.455852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95404912-22b3-4f26-ace8-79ee76fb30d4 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.456135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95404912-22b3-4f26-ace8-79ee76fb30d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.456410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95404912-22b3-4f26-ace8-79ee76fb30d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:12.456994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:12.457866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:12.458807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:12.458807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:12.458807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:12.458807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:12.458807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 334/648] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:12.459086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.470378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95404912-22b3-4f26-ace8-79ee76fb30d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-95404912-22b3-4f26-ace8-79ee76fb30d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:12.478665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:12.479606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:12.480474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 315/649] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:12.481767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95404912-22b3-4f26-ace8-79ee76fb30d4 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.502807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.503374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.575665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d785b4f-d366-4253-94d5-2b9ed2cdac35 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.589603 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95404912-22b3-4f26-ace8-79ee76fb30d4 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.616344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.617164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.617474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.617942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.624341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.625698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.625837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.626360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.626647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.626929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.627651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:12.627959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.629368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:12.640066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:12.641166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 335/650] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:12.646586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.647153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 316/651] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:12.647153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.728749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.743038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.751640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffbbb5-e89f-45f3-bdb1-93cd6a149fbb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.752712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45c33-2412-4018-b32e-f16bb1df1f8d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.787562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.788152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.788250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.788334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.788599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.788720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.788796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.788866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.789541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:12.789642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:12.789993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.790069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.791197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.791295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.800666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 317/652] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:12.802391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.805977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 336/653] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:12.807040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.913603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169892c-4b8e-43da-a435-56ad32b24955 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.914796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667aac0-059e-4766-8c36-5e3288f5d746 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:12.949267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.949920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.950205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.950483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.951272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:12.951389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:12.951579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.952076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:12.952356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:12.952668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:12.953044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.961107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:12.962982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:12.964915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:12.965501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 337/654] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:12.970953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:12.977795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:12.978751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 318/655] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:12.984548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:12.993013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:13.083544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53c6ef-f1e7-403b-900b-3b7225706a17 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.091264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3197737-d246-4172-826f-a2126c6d928c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.118781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.119568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.119850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.120125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.130053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.131482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:13.133728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.134998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.135491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.135953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.142939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.144360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:13.149411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:13.150347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:13.151276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:13.151276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:13.151276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:13.151276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:13.152841 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 338/656] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:13.153812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:13.160224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:13.161320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 319/657] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:13.167965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:13.221764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:13.229010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:13.260793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61909bbd-9536-432a-b10e-0e0f135431cd None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.277297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948280a6-0952-4253-92ec-d0d1861dc1ed None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.298429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.299109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.299394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.299674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.300578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=dea47e0ec737422c8f679e21174f8cab) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:13.300865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.302345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:13.316562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 339/658] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/role_assignments?user.id=dea47e0ec737422c8f679e21174f8cab&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:13.316562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:13.329189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.330445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.330822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.331401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.332303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:13.332644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.334902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:13.358884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 320/659] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:13.359753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:13.426412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfcc3c-66db-45d4-9098-03524541ae0a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.457162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:13.465880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4b826-f3d4-4261-97ee-55f69d5b6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.466652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b826-f3d4-4261-97ee-55f69d5b6fba None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.467016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b826-f3d4-4261-97ee-55f69d5b6fba None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.467446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b826-f3d4-4261-97ee-55f69d5b6fba None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.468266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4b826-f3d4-4261-97ee-55f69d5b6fba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:13.468674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4b826-f3d4-4261-97ee-55f69d5b6fba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.475314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4b826-f3d4-4261-97ee-55f69d5b6fba None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:13.487654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 340/660] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:13.487654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2bca0-70f3-462b-876f-d9553232fad1 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.487654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2bca0-70f3-462b-876f-d9553232fad1 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.487654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2bca0-70f3-462b-876f-d9553232fad1 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.487654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 341/661] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:13.488099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2bca0-70f3-462b-876f-d9553232fad1 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:13.496374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94172ae9-f788-4e52-a18b-ad76e95bff78 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.514169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:13.534121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.534311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.534562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.535753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.536017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:13.536328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.538579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:13.558204 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 321/662] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:13.559162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:13.613861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2bca0-70f3-462b-876f-d9553232fad1 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.652386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551656e-57ec-46d4-86f8-36fe596cbcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.653189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551656e-57ec-46d4-86f8-36fe596cbcc6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.653498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551656e-57ec-46d4-86f8-36fe596cbcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.653853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551656e-57ec-46d4-86f8-36fe596cbcc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.671440 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a354f7b-f7ce-440f-bf02-405549da185e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.707553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b07e0e-a08f-4386-aa3b-164b65d3ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.708283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07e0e-a08f-4386-aa3b-164b65d3ca2b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.708597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07e0e-a08f-4386-aa3b-164b65d3ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.708882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07e0e-a08f-4386-aa3b-164b65d3ca2b None admin] PATH_INFO: `/v3/users/glance` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.715028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:13.717249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b07e0e-a08f-4386-aa3b-164b65d3ca2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.719094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b07e0e-a08f-4386-aa3b-164b65d3ca2b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:13.728352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551656e-57ec-46d4-86f8-36fe596cbcc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.729904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551656e-57ec-46d4-86f8-36fe596cbcc6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-01b07e0e-a08f-4386-aa3b-164b65d3ca2b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:13.733481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:13.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:17:13.735386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:13.736834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 322/663] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:13.738843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb2ca-d94c-45c1-938e-62ff8015022b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.739438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb2ca-d94c-45c1-938e-62ff8015022b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.739726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb2ca-d94c-45c1-938e-62ff8015022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.757202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c1ecb2ca-d94c-45c1-938e-62ff8015022b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:13.758941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c1ecb2ca-d94c-45c1-938e-62ff8015022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:13.789816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:13.829612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 323/664] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:13.838743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb2ca-d94c-45c1-938e-62ff8015022b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:13.846839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 342/665] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:13.847886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551656e-57ec-46d4-86f8-36fe596cbcc6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:13.947754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb2ca-d94c-45c1-938e-62ff8015022b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.951839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:13.964220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551656e-57ec-46d4-86f8-36fe596cbcc6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:13.987750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:13.988413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:13.988731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:13.989015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:13.989862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=fff82e4b1a4248fcbb3566bb7bc84d26) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:13.990179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:13.991650 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.008561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 324/666] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/role_assignments?user.id=fff82e4b1a4248fcbb3566bb7bc84d26&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:14.008561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.013830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.014624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.014985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.015331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.016299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:14.016790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.018373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.044175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 343/667] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:14.045420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.047618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:14.157592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa2fa1-0052-468f-9ff7-3e978ecc0897 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.180538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ab12d-890a-4014-b73d-ff67ebeb6997 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.203470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.204270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.204641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.205403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.218771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.220153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.228470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fa949-a676-436b-a3f6-4f6baf26c57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.229326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa949-a676-436b-a3f6-4f6baf26c57a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.229617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa949-a676-436b-a3f6-4f6baf26c57a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.229902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa949-a676-436b-a3f6-4f6baf26c57a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.241935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fa949-a676-436b-a3f6-4f6baf26c57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:14.243143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:14.244242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:14.245470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:14.245470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:14.245470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:14.245470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:14.247310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 325/668] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:14.248262 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-fd0fa949-a676-436b-a3f6-4f6baf26c57a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:14.254633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:14.255652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:14.256650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 344/669] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:14.257416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fa949-a676-436b-a3f6-4f6baf26c57a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.267884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:14.370042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c857e-696c-46ad-a9e2-31faf249cddd None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.380730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fa949-a676-436b-a3f6-4f6baf26c57a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.405083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5164-fb42-4f9a-8a3d-e37079f9cbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.405798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5164-fb42-4f9a-8a3d-e37079f9cbeb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.406088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5164-fb42-4f9a-8a3d-e37079f9cbeb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.406424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5164-fb42-4f9a-8a3d-e37079f9cbeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.415566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.416239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.416546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.416823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.417543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:14.417913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.419275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.433750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 345/670] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:14.434634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.437244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:14.481754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5164-fb42-4f9a-8a3d-e37079f9cbeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.483435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5164-fb42-4f9a-8a3d-e37079f9cbeb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.513770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:14.544402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3d81e-9760-4538-8c89-28ee1a10cac4 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.582234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:14.596510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.597538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.597841 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.598131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.599170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:14.599489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.601066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.611620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 326/671] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:14.611620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5164-fb42-4f9a-8a3d-e37079f9cbeb None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.618498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 346/672] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:14.619427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.728152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a07a71-4d80-47ba-aece-f3f7c552d838 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.731884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5164-fb42-4f9a-8a3d-e37079f9cbeb None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.757218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:14.766376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.767179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.767474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.767759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.770345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.771382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.771918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.772402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.778834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.785866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.785989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.786070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:14.797247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:14.798838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:14.799758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:14.799758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:14.799758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:14.799758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:14.799758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:14.799758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:14.799758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:14.799758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:14.800238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:14.801651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:14.802895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:14.802895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:14.802895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:14.802895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:14.804222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 347/673] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:14.805763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.806255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 327/674] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:14.807485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.852560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:14.911334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4582a-4e10-4419-9969-24fde6c2af60 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.915936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460014a-e7a6-4abd-a10c-435fed80ff9d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:14.948144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.949219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.949855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.950347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.951298 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:14.952011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.953085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:14.953881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:14.954123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.954237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:14.954429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:14.966472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:14.967903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 328/675] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:14.968015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:14.969118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:14.991379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:14.992364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:14.993324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:14.993324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:14.993324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:14.993324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:14.993324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 348/676] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:14.994054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:15.028120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:15.075791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15b676-e137-45ae-9716-7f96a6b76e15 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:15.077828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:15.104448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf23dd-5df0-4681-bca6-a969a89cc4db None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:15.112645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:15.113228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.113818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.114200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.115285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:15.115572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:15.117254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:15.129799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 329/677] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:15.130773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:15.143550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:15.144600 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.145160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.145632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.157622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:15.159396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:15.175619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:15.176607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:15.177561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:15.177561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:15.177561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:15.177561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:15.177561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:15.177561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:15.177561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:15.177561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:15.184585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 349/678] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:15.185767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:15.244348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f67be-0fab-4f7c-be3e-7ca1cf6f15ee None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:15.279491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:15.280305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.280614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.280964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.281766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:15.282140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:15.283664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:15.292159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:15.297871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 330/679] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:15.299126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:15.303494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8758a93-d72e-43de-99d7-bc8c76405d5b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:15.324841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:15.341823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:15.342798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.343284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.343746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.344866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=96df1ee2cbc14bcc8a8a250fd8f349dd) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:15.345400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:15.347070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:15.353851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 350/680] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:15 2022] GET /identity/v3/role_assignments?user.id=96df1ee2cbc14bcc8a8a250fd8f349dd&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:15.354947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:15.413090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0969a-7e70-435b-b649-d2891f12f052 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:15.457736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:15.458443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.458776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.459059 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.459967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:15.460285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:15.462479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:15.471111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0fd60-b0b3-4f0c-a58e-9d602c99143c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:15.482026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 331/681] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:15.494586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:15.553320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7822b-a953-4f33-b49f-7550317217f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:15.554250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7822b-a953-4f33-b49f-7550317217f6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.554532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7822b-a953-4f33-b49f-7550317217f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.555552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7822b-a953-4f33-b49f-7550317217f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.574549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7822b-a953-4f33-b49f-7550317217f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:15.576013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7822b-a953-4f33-b49f-7550317217f6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-49b7822b-a953-4f33-b49f-7550317217f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:15.603624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:15.606795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:15.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 351/682] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:15.609017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfda76-8d7e-42c8-8a6e-a47579b7fb21 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.609318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfda76-8d7e-42c8-8a6e-a47579b7fb21 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.609606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfda76-8d7e-42c8-8a6e-a47579b7fb21 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.611287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 352/683] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:15.615516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfda76-8d7e-42c8-8a6e-a47579b7fb21 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:15.639129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:15.646456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:15.670054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca34eaa-2496-4f65-a526-4055bb86f690 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:15.714500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae74b0-7581-4b3b-909e-a29f925c30a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:15.715520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae74b0-7581-4b3b-909e-a29f925c30a5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.716016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae74b0-7581-4b3b-909e-a29f925c30a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.716570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae74b0-7581-4b3b-909e-a29f925c30a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.717667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae74b0-7581-4b3b-909e-a29f925c30a5 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=c958b233374b4013a8bbe9efbed054a2) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:15.718199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae74b0-7581-4b3b-909e-a29f925c30a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:15.720370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae74b0-7581-4b3b-909e-a29f925c30a5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:15.729909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 332/684] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:15 2022] GET /identity/v3/role_assignments?user.id=c958b233374b4013a8bbe9efbed054a2&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:15.731904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b43aae-306d-4369-ac11-60b68082118d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.732387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b43aae-306d-4369-ac11-60b68082118d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.732871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b43aae-306d-4369-ac11-60b68082118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.761182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfda76-8d7e-42c8-8a6e-a47579b7fb21 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:15.765407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-26b43aae-306d-4369-ac11-60b68082118d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:15.769414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-26b43aae-306d-4369-ac11-60b68082118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:15.846074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:15.846770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:15.847052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:15.847330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:15.848289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:15.848626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:15.850198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:15.875967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 333/685] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:15.888386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 353/686] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jun 16 00:17:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:15.894159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b43aae-306d-4369-ac11-60b68082118d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:15.909038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.038331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:16.042686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:16.058493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06525a-23a4-4d71-bbc8-eb90f1ee2ae4 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.071313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b43aae-306d-4369-ac11-60b68082118d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.110532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c1259-18ff-4c1c-9b85-d8929e3bc18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.111963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1259-18ff-4c1c-9b85-d8929e3bc18d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.112258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1259-18ff-4c1c-9b85-d8929e3bc18d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.112574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1259-18ff-4c1c-9b85-d8929e3bc18d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.116549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500657a8-315b-475b-bc7f-d8e8187dfb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.117277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500657a8-315b-475b-bc7f-d8e8187dfb1d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.118202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500657a8-315b-475b-bc7f-d8e8187dfb1d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.118481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500657a8-315b-475b-bc7f-d8e8187dfb1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.125679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c1259-18ff-4c1c-9b85-d8929e3bc18d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-7e4c1259-18ff-4c1c-9b85-d8929e3bc18d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:16.135076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:16.136050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:16.136937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:16.137608 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 354/687] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:16.144143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c1259-18ff-4c1c-9b85-d8929e3bc18d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.201708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500657a8-315b-475b-bc7f-d8e8187dfb1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:16.203182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500657a8-315b-475b-bc7f-d8e8187dfb1d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:16.254910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c1259-18ff-4c1c-9b85-d8929e3bc18d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.287136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:16.300411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.300411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.300411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.300411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.301198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:16.301198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:16.301198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:16.301198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:16.308626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 355/688] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:16.314714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.318210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 334/689] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:16.327668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500657a8-315b-475b-bc7f-d8e8187dfb1d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.437731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a23e2-c9d7-4c0f-bff2-026e0132a973 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.450752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500657a8-315b-475b-bc7f-d8e8187dfb1d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.479382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323e77e-8ad3-4eed-8a11-72f8babbce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.480815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323e77e-8ad3-4eed-8a11-72f8babbce41 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.481215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323e77e-8ad3-4eed-8a11-72f8babbce41 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.481767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323e77e-8ad3-4eed-8a11-72f8babbce41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.487968 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.488844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.489125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.489403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.494308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323e77e-8ad3-4eed-8a11-72f8babbce41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:16.495779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323e77e-8ad3-4eed-8a11-72f8babbce41 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:16.499777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:16.501268 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-5323e77e-8ad3-4eed-8a11-72f8babbce41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:16.516359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:16.518431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:16.520583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:16.520583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:16.520583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:16.520583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:16.520583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 356/690] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:16.520583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071293c6-486e-4ba6-ad21-ee7997d63b26 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.520583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071293c6-486e-4ba6-ad21-ee7997d63b26 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.520583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071293c6-486e-4ba6-ad21-ee7997d63b26 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.521655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 357/691] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:16.526587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:16.528480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:16.530699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:16.530699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:16.530699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:16.530699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:16.531183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d3ad-8550-4277-ad71-e75100416aa4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.531183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d3ad-8550-4277-ad71-e75100416aa4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.531521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 335/692] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:16.531668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d3ad-8550-4277-ad71-e75100416aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.536545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.548538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-eed8d3ad-8550-4277-ad71-e75100416aa4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:16.549751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-eed8d3ad-8550-4277-ad71-e75100416aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:16.569035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:16.605443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:16.630781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 358/693] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:16.631934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8d3ad-8550-4277-ad71-e75100416aa4 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.635704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fe58f-20e9-49f0-a270-ab26fe9c043b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.671120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.671801 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.672089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.672371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.673463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:16.673938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:16.675300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:16.688543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 336/694] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:16.693453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.744303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8d3ad-8550-4277-ad71-e75100416aa4 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.796560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.797386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.797773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.798149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.799723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:16.800134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:16.802369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:16.812489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:16.821108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 359/695] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:16.821990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.830700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021092c-f89f-4e37-bb78-3acf1e7c4241 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.871825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:16.882576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.884751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.885740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.886283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.887427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:16.889783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:16.910004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:16.911051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:16.911914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:16.918775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 337/696] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Jun 16 00:17:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:16.918775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:16.950485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101a657-9404-4487-95f2-448e9a1fd194 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:16.989499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:16.990467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:16.990864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:16.991242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:16.999949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.001529 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:17.015918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:17.017146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:17.018322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:17.024540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 360/697] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:17:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:17.024540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:17.047554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82e9a9-5798-4f27-b752-2d98567a88ef None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:17.088795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:17.089742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.090124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.090526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.097882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.099494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:17.114950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:17.116040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:17.117224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:17.117224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:17.117224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:17.117224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:17.117224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:17.117224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:17.117224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:17.117224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:17.123888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 338/698] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:17.123888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:17.137224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:17.140978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74d868-2056-458f-ba18-e21deb687d30 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:17.145859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:17.177574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:17.178625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.178914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.179250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.180665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:17.180977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.182428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.188615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 361/699] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jun 16 00:17:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:17.188615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:17.246422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376714a2-8bb2-42b2-9ff7-2c6548c30a4c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:17.283559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:17.284229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.284509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.284808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.285866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:17.286180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.287615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.292449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3e61-36e7-4591-9c6d-d431153a0c03 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:17.313390 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 339/700] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:17:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:17.314295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:17.330456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851028d-f854-44d1-8551-f20179558013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:17.331170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851028d-f854-44d1-8551-f20179558013 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.331461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851028d-f854-44d1-8551-f20179558013 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.331730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851028d-f854-44d1-8551-f20179558013 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.332556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851028d-f854-44d1-8551-f20179558013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:17.332914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851028d-f854-44d1-8551-f20179558013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.334348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851028d-f854-44d1-8551-f20179558013 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.347686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 362/701] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:17.348599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851028d-f854-44d1-8551-f20179558013 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:17.374875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:17.395116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:17.429474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccffaae-83cf-4b1f-babe-4be0a99bb93e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:17.454478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851028d-f854-44d1-8551-f20179558013 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:17.469264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:17.470038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.470323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.470596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.471309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:17.471613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.473277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.484836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 340/702] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jun 16 00:17:17 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:17.485920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:17.491326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c55682-057b-44b0-990a-e3bfaa2c83d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:17.491986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55682-057b-44b0-990a-e3bfaa2c83d4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.492356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55682-057b-44b0-990a-e3bfaa2c83d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.492666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55682-057b-44b0-990a-e3bfaa2c83d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.567728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c55682-057b-44b0-990a-e3bfaa2c83d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.569289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c55682-057b-44b0-990a-e3bfaa2c83d4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.608665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:17.640187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491e033-7249-4029-8c55-9d16a78aebaa None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:17.682420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:17.683289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.683689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.684161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.685272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=645bbe41756749b98232d0e2eb2e678a) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:17.685592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.687195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.689418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:17.695511 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 341/703] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:17 2022] GET /identity/v3/role_assignments?user.id=645bbe41756749b98232d0e2eb2e678a&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:17.730885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 363/704] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:17:17.737616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:17.868994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434b92a-c762-4ffd-b6f9-481a0107c951 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:17.906380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:17.907004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136214c-55fa-487a-b228-9e097cbde231 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.907271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136214c-55fa-487a-b228-9e097cbde231 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.907490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136214c-55fa-487a-b228-9e097cbde231 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.908449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.908893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.909302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.909421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 364/705] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:17.914674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc2f3-e0f4-4897-adc0-b8b4c60cd055 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:17.915072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc2f3-e0f4-4897-adc0-b8b4c60cd055 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:17.915461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc2f3-e0f4-4897-adc0-b8b4c60cd055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:17.921969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:17.923877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:17.933483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f26dc2f3-e0f4-4897-adc0-b8b4c60cd055 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:17.934807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f26dc2f3-e0f4-4897-adc0-b8b4c60cd055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:17.942277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:17.943375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:17.944470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:17.944470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:17.944470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:17.944470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:17.946916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 342/706] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:17.952351 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.004987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:18.028453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 365/707] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:17:18.051639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dc2f3-e0f4-4897-adc0-b8b4c60cd055 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.061044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd85cc4-f322-4456-ae9b-0727c1ee98b5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:18.097880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:18.098799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.099232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.099654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.100563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:18.101031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:18.102606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:18.114045 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:18.120682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 343/708] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:18.122674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.165396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dc2f3-e0f4-4897-adc0-b8b4c60cd055 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:18.204665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:18.205637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.206018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.206603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] PATH_INFO: `/v3/services/network` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.207166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:18.209337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:18.223470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:18.224585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:18.225446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 366/709] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:18.231373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.234283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517fca4-cbc0-4b52-8752-5e3930fe6ec4 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:18.246384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:18.271306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:18.272401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.272946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.273397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.280625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:18.282394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:18.297522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:18.298759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:18.299685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:18.299685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:18.299685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:18.299685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:18.299685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:18.299685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:18.299685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:18.299685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:18.302172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 344/710] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:18.308353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.342924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d07b02-4069-4962-bfce-c4dcabf87831 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:18.360842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:18.388617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:18.388617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.388617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.388617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.389004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] RBAC: Adding query filter params (name=network) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:18.389004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:18.389004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:18.392935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 367/711] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:18.394319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.417022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5ffed-44d5-4747-b06f-bc4ce967f9a5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:18.453946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:18.454964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.455447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.455919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.456947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:18.457500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:18.459113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:18.472983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 345/712] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:18.479850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.507696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ea892-48a1-48b3-b286-bcc17883d99f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:18.514143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:18.544964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c89ab-b982-4ff2-93d1-c949c5042c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:18.545365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c89ab-b982-4ff2-93d1-c949c5042c13 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.545656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c89ab-b982-4ff2-93d1-c949c5042c13 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.545940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c89ab-b982-4ff2-93d1-c949c5042c13 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.546676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c89ab-b982-4ff2-93d1-c949c5042c13 None admin] RBAC: Adding query filter params (type=network) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:18.546965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c89ab-b982-4ff2-93d1-c949c5042c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:18.548653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c89ab-b982-4ff2-93d1-c949c5042c13 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:18.556591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 368/713] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:18.557830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7ade-75ed-450e-b157-e0463e6ae64c None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.558126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7ade-75ed-450e-b157-e0463e6ae64c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.558412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7ade-75ed-450e-b157-e0463e6ae64c None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.566051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 369/714] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:18.571597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccabb4-ef44-4d4a-8c77-279cf4960b0d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.571942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccabb4-ef44-4d4a-8c77-279cf4960b0d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.572164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccabb4-ef44-4d4a-8c77-279cf4960b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.592920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e389a34-7e5c-4c3a-b493-87811410f7e5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:18.606428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1bccabb4-ef44-4d4a-8c77-279cf4960b0d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:18.607779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1bccabb4-ef44-4d4a-8c77-279cf4960b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:18.633857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:18.634913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.635362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.635877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.637136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=dea47e0ec737422c8f679e21174f8cab) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:18.637692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:18.639653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:18.646682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 346/715] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/role_assignments?user.id=dea47e0ec737422c8f679e21174f8cab&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:18.687218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:18.738413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 370/716] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:18.773368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.888252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:18.894983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b7b19-cabc-4ec4-9547-1daae6c1aed8 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:18.936736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:18.938383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.938867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.939355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.951095 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:18.952814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:18.970789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:18.971771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:18.972718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:18.972718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:18.972718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:18.972718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:18.975068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 347/717] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:18.980543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccabb4-ef44-4d4a-8c77-279cf4960b0d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:19.093569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccabb4-ef44-4d4a-8c77-279cf4960b0d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:19.133619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d90b55-f75a-44cb-b74f-69e5ac6c1109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:19.134299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d90b55-f75a-44cb-b74f-69e5ac6c1109 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:19.134587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d90b55-f75a-44cb-b74f-69e5ac6c1109 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:19.134870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d90b55-f75a-44cb-b74f-69e5ac6c1109 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:19.135610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d90b55-f75a-44cb-b74f-69e5ac6c1109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:19.135925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d90b55-f75a-44cb-b74f-69e5ac6c1109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:19.137332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d90b55-f75a-44cb-b74f-69e5ac6c1109 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:19.149798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 371/718] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:19.154830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:19.267804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85a129-112f-4145-bdbb-2d9d98a5f69a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:19.302773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b1820-cc13-4696-aeb0-f3996057370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:19.303839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b1820-cc13-4696-aeb0-f3996057370f None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:19.304332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b1820-cc13-4696-aeb0-f3996057370f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:19.305474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b1820-cc13-4696-aeb0-f3996057370f None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:19.306378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b1820-cc13-4696-aeb0-f3996057370f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:19.308005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b1820-cc13-4696-aeb0-f3996057370f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:19.405167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 348/719] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:17:19 2022] POST /identity/v3/users => generated 254 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:20.197211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aafe2-4f65-46b3-9e09-ead2209690d7 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:20.197732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aafe2-4f65-46b3-9e09-ead2209690d7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:20.197811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aafe2-4f65-46b3-9e09-ead2209690d7 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:20.199394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 372/720] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:20.204662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802519e-0d91-4f19-9827-94b8e2f5bd67 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:20.205461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802519e-0d91-4f19-9827-94b8e2f5bd67 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:20.205961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802519e-0d91-4f19-9827-94b8e2f5bd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:20.211774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:20.223455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-2802519e-0d91-4f19-9827-94b8e2f5bd67 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:20.224671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2802519e-0d91-4f19-9827-94b8e2f5bd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:20.301183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 349/721] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:20.336304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aafe2-4f65-46b3-9e09-ead2209690d7 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:20.381652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:20.465455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aafe2-4f65-46b3-9e09-ead2209690d7 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:20.516653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c257f5-ff96-41dd-bd5b-5953487e8ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:20.517329 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c257f5-ff96-41dd-bd5b-5953487e8ed9 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:20.517614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c257f5-ff96-41dd-bd5b-5953487e8ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:20.517894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c257f5-ff96-41dd-bd5b-5953487e8ed9 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:20.518630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c257f5-ff96-41dd-bd5b-5953487e8ed9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:20.520006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c257f5-ff96-41dd-bd5b-5953487e8ed9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:20.648864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 373/722] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:20 2022] POST /identity/v3/services => generated 251 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:20.918432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb49945-fbc9-4432-a245-b58b663c0f09 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:20.918971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb49945-fbc9-4432-a245-b58b663c0f09 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:20.919423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb49945-fbc9-4432-a245-b58b663c0f09 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:20.921285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 350/723] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:20.926748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ede7fe-bf2d-4592-b0d2-d1f3c55ea1d4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:20.927042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ede7fe-bf2d-4592-b0d2-d1f3c55ea1d4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:20.927324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ede7fe-bf2d-4592-b0d2-d1f3c55ea1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:20.947743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a4ede7fe-bf2d-4592-b0d2-d1f3c55ea1d4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:20.948996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a4ede7fe-bf2d-4592-b0d2-d1f3c55ea1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:21.028773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 374/724] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:21.052100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb49945-fbc9-4432-a245-b58b663c0f09 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:21.168876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb49945-fbc9-4432-a245-b58b663c0f09 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:21.191561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c243d7-4e9d-4cbd-b86d-a7d598c7b4ce None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.191880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c243d7-4e9d-4cbd-b86d-a7d598c7b4ce None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.192129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c243d7-4e9d-4cbd-b86d-a7d598c7b4ce None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.193785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 375/725] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:21.195122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b3e8d-cdda-4ab3-9a99-c39bd5a62904 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.196166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b3e8d-cdda-4ab3-9a99-c39bd5a62904 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.197252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b3e8d-cdda-4ab3-9a99-c39bd5a62904 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.202966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 376/726] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:21.202966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e4500-aa33-4f63-9378-eab417bcc55a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.202966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e4500-aa33-4f63-9378-eab417bcc55a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.203278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e4500-aa33-4f63-9378-eab417bcc55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.203953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:21.209012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c36a5b-e45f-4d94-ad46-bd291313ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:21.209811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c36a5b-e45f-4d94-ad46-bd291313ab7b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.210139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c36a5b-e45f-4d94-ad46-bd291313ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.210361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c36a5b-e45f-4d94-ad46-bd291313ab7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.220835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c36a5b-e45f-4d94-ad46-bd291313ab7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:21.222330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-dc4e4500-aa33-4f63-9378-eab417bcc55a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:21.223530 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dc4e4500-aa33-4f63-9378-eab417bcc55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-c8c36a5b-e45f-4d94-ad46-bd291313ab7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:21.228835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:21.229794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:21.230653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 351/727] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:21.231500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb424b8a-d31c-41e9-9274-8b58e8b8b737 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.231930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb424b8a-d31c-41e9-9274-8b58e8b8b737 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.232355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb424b8a-d31c-41e9-9274-8b58e8b8b737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.243756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:21.249821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-fb424b8a-d31c-41e9-9274-8b58e8b8b737 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:21.250996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fb424b8a-d31c-41e9-9274-8b58e8b8b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:21.304312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 377/728] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:21.305977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e4500-aa33-4f63-9378-eab417bcc55a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:21.325259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 352/729] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:17:21.327950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb424b8a-d31c-41e9-9274-8b58e8b8b737 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:21.429549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e4500-aa33-4f63-9378-eab417bcc55a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:21.441017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb424b8a-d31c-41e9-9274-8b58e8b8b737 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:21.465991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:21.466954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.467420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.467906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.468864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:21.469357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:21.470892 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:21.472933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:21.480434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 378/730] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:21.481454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:21.485382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271690f-a0d4-4eb8-b5c6-181ce1060676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:21.486201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271690f-a0d4-4eb8-b5c6-181ce1060676 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.486485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271690f-a0d4-4eb8-b5c6-181ce1060676 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.486761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271690f-a0d4-4eb8-b5c6-181ce1060676 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.501909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271690f-a0d4-4eb8-b5c6-181ce1060676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:21.502660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-9271690f-a0d4-4eb8-b5c6-181ce1060676 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:21.512427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:21.513470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:21.514358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:21.517877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 353/731] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:21.519059 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271690f-a0d4-4eb8-b5c6-181ce1060676 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:21.606915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524c8a-821c-49e7-a47c-73ea3243cfc7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:21.631058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271690f-a0d4-4eb8-b5c6-181ce1060676 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:21.651653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454ae6-1500-4f6f-94c3-05f017a9c790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:21.652556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454ae6-1500-4f6f-94c3-05f017a9c790 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.653340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454ae6-1500-4f6f-94c3-05f017a9c790 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.653632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454ae6-1500-4f6f-94c3-05f017a9c790 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.654644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b454ae6-1500-4f6f-94c3-05f017a9c790 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:21.656180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b454ae6-1500-4f6f-94c3-05f017a9c790 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:21.668248 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:21.669448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.669911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.670350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.683294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:21.684972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:21.702745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:21.703825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:21.704946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:21.704946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:21.704946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:21.704946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:21.706450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 354/732] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:21.707432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:21.732187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 379/733] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:21 2022] POST /identity/v3/services => generated 240 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:21.733772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454ae6-1500-4f6f-94c3-05f017a9c790 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:21.761386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:21.817189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:21.867657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454ae6-1500-4f6f-94c3-05f017a9c790 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:21.885202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1bd0-47cb-408a-99b8-0c37aa2fc4a0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:21.942848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:21.943461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.944313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.944626 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.945983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:21.946301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:21.947687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:21.951325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:21.952011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:21.952302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:21.952628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:21.953371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:21.953688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:21.955097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:21.972362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 355/734] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:21.976273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 380/735] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:21.979917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:21.983752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.102928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80796443-01e2-46f3-8325-af4a8914d6cc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.112973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.147545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.149384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.149678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.149959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.175317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.176798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:22.201846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:22.202780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:22.203692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:22.203692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:22.203692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:22.203692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:22.214219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 356/736] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:22.215117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.266921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c67d8d-95a8-4435-91d8-398d2024dddb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.309379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.330214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.331023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.331306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.331582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.343919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.350802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b9832-0108-4f77-863f-20f5c2292633 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.354289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:22.370793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:22.372006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:22.373308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 381/737] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:22.379685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.433560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b737cd-6442-47cc-853d-eea4152223d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.434251 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b737cd-6442-47cc-853d-eea4152223d9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.434537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b737cd-6442-47cc-853d-eea4152223d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.434819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b737cd-6442-47cc-853d-eea4152223d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.435559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b737cd-6442-47cc-853d-eea4152223d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:22.435868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b737cd-6442-47cc-853d-eea4152223d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.437258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b737cd-6442-47cc-853d-eea4152223d9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.456608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 357/738] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:22.457287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dc340-78fb-4766-b01b-f5454e0baae2 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.457589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dc340-78fb-4766-b01b-f5454e0baae2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.457869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dc340-78fb-4766-b01b-f5454e0baae2 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.459732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 358/739] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:22.460505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155dc340-78fb-4766-b01b-f5454e0baae2 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.472419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.502065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99e9b9-1121-4ea4-8595-2b72e401608c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.537182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7494c-7be2-4908-9478-aefab485caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.537859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7494c-7be2-4908-9478-aefab485caae None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.538146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7494c-7be2-4908-9478-aefab485caae None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.538424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7494c-7be2-4908-9478-aefab485caae None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.540247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7494c-7be2-4908-9478-aefab485caae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:22.540247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7494c-7be2-4908-9478-aefab485caae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.541600 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7494c-7be2-4908-9478-aefab485caae None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.559304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 382/740] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:22.560712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5877b8-77f3-4697-8e2a-1e987b870c3c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.561009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5877b8-77f3-4697-8e2a-1e987b870c3c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.561297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5877b8-77f3-4697-8e2a-1e987b870c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.572775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155dc340-78fb-4766-b01b-f5454e0baae2 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.581528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-8d5877b8-77f3-4697-8e2a-1e987b870c3c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:22.582739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8d5877b8-77f3-4697-8e2a-1e987b870c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:22.621333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.622483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.623027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.623518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.626695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.632762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.634525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:22.651195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:22.652645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:22.653657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:22.654226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 383/741] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:22.654226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5877b8-77f3-4697-8e2a-1e987b870c3c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.660542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 359/742] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:22.662041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.736964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.770147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5877b8-77f3-4697-8e2a-1e987b870c3c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.776305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469aad1-b9ca-4ead-8847-f1a2abb5315b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.806061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8e785-d340-416d-bc5b-6432ca4c630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.806790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8e785-d340-416d-bc5b-6432ca4c630e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.806995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8e785-d340-416d-bc5b-6432ca4c630e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.807277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8e785-d340-416d-bc5b-6432ca4c630e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.819125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.820749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.821270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.821751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.822815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:22.823456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.825261 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.849882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 360/743] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:22.851238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.861426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.885054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8e785-d340-416d-bc5b-6432ca4c630e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.886557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8e785-d340-416d-bc5b-6432ca4c630e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.963787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fab42f-6510-4bba-ac28-570e143420ee None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.988645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 384/744] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:22.989441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8e785-d340-416d-bc5b-6432ca4c630e None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.995381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.005338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8858e-93ff-43d7-acd4-e66fc57a8259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.006391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8858e-93ff-43d7-acd4-e66fc57a8259 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.012713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8858e-93ff-43d7-acd4-e66fc57a8259 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.012713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8858e-93ff-43d7-acd4-e66fc57a8259 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.018912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8858e-93ff-43d7-acd4-e66fc57a8259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-90e8858e-93ff-43d7-acd4-e66fc57a8259 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:23.027363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:23.028339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:23.029533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:23.032241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 361/745] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:23.033399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8858e-93ff-43d7-acd4-e66fc57a8259 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.071838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.095022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8e785-d340-416d-bc5b-6432ca4c630e None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.131333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.132124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.132409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.132745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.143149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8858e-93ff-43d7-acd4-e66fc57a8259 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.143584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.145186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:23.164734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:23.165714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:23.166668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:23.166668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:23.166668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:23.166668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:23.170656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 385/746] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:23.171644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.186639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.187764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.188354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.188887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.200926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.202973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:23.225217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:23.227136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:23.228064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:23.228064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:23.228064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:23.228064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:23.230614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 362/747] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:23.232158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.282335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f427c5-336f-4340-8b19-8ab7127eca11 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.284777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.316144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.325355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.326060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.327074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.327074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.328204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:23.328556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.330121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.345115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 386/748] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:23.345115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.363384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798eff38-a6c8-4ef8-a1ce-39dd37d6ebf9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.401664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.403144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.403908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.404398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.405521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:23.406318 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.408257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.428185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 363/749] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:23.428185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.451280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a65e0-b2e3-46b7-a155-8d5383151332 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.487949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118256-b3a6-480a-9635-3020df3074ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.488636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118256-b3a6-480a-9635-3020df3074ba None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.488957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118256-b3a6-480a-9635-3020df3074ba None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.489237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118256-b3a6-480a-9635-3020df3074ba None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.489970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118256-b3a6-480a-9635-3020df3074ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:23.490270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118256-b3a6-480a-9635-3020df3074ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.491634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118256-b3a6-480a-9635-3020df3074ba None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.503395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 387/750] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:23.504697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d21ed-8439-41b5-bbdb-6d51dbac0012 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.504931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d21ed-8439-41b5-bbdb-6d51dbac0012 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.505207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d21ed-8439-41b5-bbdb-6d51dbac0012 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.506544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 388/751] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:23.511234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d21ed-8439-41b5-bbdb-6d51dbac0012 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.531102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.534942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d9d0e-b1ab-47e3-91f3-ab00a03b032b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.566510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.571314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.572110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.572395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.572725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.583022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.584485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:23.606728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:23.607674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:23.608652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:23.608652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:23.608652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:23.608652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:23.608652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 364/752] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:23.608934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.634374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d21ed-8439-41b5-bbdb-6d51dbac0012 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.671828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120f9e4-88c1-4a9e-978a-0abf21cfce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.672881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120f9e4-88c1-4a9e-978a-0abf21cfce71 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.673598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120f9e4-88c1-4a9e-978a-0abf21cfce71 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.673934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120f9e4-88c1-4a9e-978a-0abf21cfce71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.680684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120f9e4-88c1-4a9e-978a-0abf21cfce71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.682119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120f9e4-88c1-4a9e-978a-0abf21cfce71 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-d120f9e4-88c1-4a9e-978a-0abf21cfce71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:23.696304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:23.697293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:23.698276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 389/753] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:23.698789 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a98e9-2c15-40eb-b730-458cbd5c6e2f None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.699341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a98e9-2c15-40eb-b730-458cbd5c6e2f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.699341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a98e9-2c15-40eb-b730-458cbd5c6e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.718464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-885a98e9-2c15-40eb-b730-458cbd5c6e2f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:23.718910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3a573-a151-425e-aa34-2ff159da8667 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.719650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-885a98e9-2c15-40eb-b730-458cbd5c6e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:23.754845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.756151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.756151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.756151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.763662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.765128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:23.779667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:23.780640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:23.781569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 365/754] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:23.782071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.795252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 390/755] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:23.795766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.796034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a98e9-2c15-40eb-b730-458cbd5c6e2f None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.841404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.903932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a98e9-2c15-40eb-b730-458cbd5c6e2f None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.917524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53e45e-3dc0-447c-a2a2-91f330072f3a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.941859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33569757-de7a-406c-bd12-513a86494492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.942534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569757-de7a-406c-bd12-513a86494492 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.942826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569757-de7a-406c-bd12-513a86494492 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.943111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569757-de7a-406c-bd12-513a86494492 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.943904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33569757-de7a-406c-bd12-513a86494492 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:23.944218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33569757-de7a-406c-bd12-513a86494492 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.946225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33569757-de7a-406c-bd12-513a86494492 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.956751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.957748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.958230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.958376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 391/756] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:23.958810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.959243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33569757-de7a-406c-bd12-513a86494492 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.959950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:23.960263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.962238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.974927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 366/757] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:23.975742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.063246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.074939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.078084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33569757-de7a-406c-bd12-513a86494492 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.098111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cef37-64a5-47c4-a5fd-bbec1359b497 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.117655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.119013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.119618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.120171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.121343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:24.122144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.124156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.132122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.133525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.133811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.134089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.134744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.136101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.138489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 392/758] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:24.139193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:24.153486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:24.154540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:24.155394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 367/759] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:24.156271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.250296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23f1f5-8100-4f8c-b0e5-792aba035579 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.261549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54734bd6-2c72-4686-b30e-951318450925 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.283285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2543b-f2b8-4a72-84f5-b2504a1c5e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.284161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2543b-f2b8-4a72-84f5-b2504a1c5e29 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.284442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2543b-f2b8-4a72-84f5-b2504a1c5e29 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.284748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2543b-f2b8-4a72-84f5-b2504a1c5e29 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/dea47e0ec737422c8f679e21174f8cab/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.286671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2543b-f2b8-4a72-84f5-b2504a1c5e29 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=dea47e0ec737422c8f679e21174f8cab, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.288092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2543b-f2b8-4a72-84f5-b2504a1c5e29 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.293560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f050-b04f-4119-8184-851a8339974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.294363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f050-b04f-4119-8184-851a8339974a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.294649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f050-b04f-4119-8184-851a8339974a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.294929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f050-b04f-4119-8184-851a8339974a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.296875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.302729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619f050-b04f-4119-8184-851a8339974a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.304143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619f050-b04f-4119-8184-851a8339974a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-7619f050-b04f-4119-8184-851a8339974a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:24.318865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:24.319972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 368/760] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:24.321858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f050-b04f-4119-8184-851a8339974a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.360675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 393/761] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:24 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/dea47e0ec737422c8f679e21174f8cab/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:24.361492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2543b-f2b8-4a72-84f5-b2504a1c5e29 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.387338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.437646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f050-b04f-4119-8184-851a8339974a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.473928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2543b-f2b8-4a72-84f5-b2504a1c5e29 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.477606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a68bb-7635-4acd-90d1-bd3766b96a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.478294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a68bb-7635-4acd-90d1-bd3766b96a6f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.478579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a68bb-7635-4acd-90d1-bd3766b96a6f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.478855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a68bb-7635-4acd-90d1-bd3766b96a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.479706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a68bb-7635-4acd-90d1-bd3766b96a6f None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=645bbe41756749b98232d0e2eb2e678a) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:24.480023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a68bb-7635-4acd-90d1-bd3766b96a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.481510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a68bb-7635-4acd-90d1-bd3766b96a6f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.488274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 369/762] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/role_assignments?user.id=645bbe41756749b98232d0e2eb2e678a&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:24.489486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013614c-e160-41b0-9317-0f688d6c611d None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.489775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013614c-e160-41b0-9317-0f688d6c611d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.490056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013614c-e160-41b0-9317-0f688d6c611d None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.491513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 370/763] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:24.496587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013614c-e160-41b0-9317-0f688d6c611d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.531523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922177c-ee15-482e-8bb8-c3f00dd01a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.531523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922177c-ee15-482e-8bb8-c3f00dd01a6e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.531523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922177c-ee15-482e-8bb8-c3f00dd01a6e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.531864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922177c-ee15-482e-8bb8-c3f00dd01a6e None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.532555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922177c-ee15-482e-8bb8-c3f00dd01a6e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:24.532877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922177c-ee15-482e-8bb8-c3f00dd01a6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.534268 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922177c-ee15-482e-8bb8-c3f00dd01a6e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.540488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 394/764] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:24.542873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba47ff3-820e-4d02-9890-0ffb3bbebd14 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.542873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba47ff3-820e-4d02-9890-0ffb3bbebd14 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.542873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba47ff3-820e-4d02-9890-0ffb3bbebd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.566518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-8ba47ff3-820e-4d02-9890-0ffb3bbebd14 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:24.567746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8ba47ff3-820e-4d02-9890-0ffb3bbebd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:24.575223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.644895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013614c-e160-41b0-9317-0f688d6c611d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.659744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 395/765] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:24.660741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba47ff3-820e-4d02-9890-0ffb3bbebd14 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.682236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.689889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.690573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.690853 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.691200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.692504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:24.692893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.696376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.765081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 371/766] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:24.765997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.832785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba47ff3-820e-4d02-9890-0ffb3bbebd14 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.875292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.875963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.876409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.876721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.878157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:24.878478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.880306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.887717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5df055-240b-43c9-ae8d-3a0deacc6fa2 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.892151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 396/767] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:24.893953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.910236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.930373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.931422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.932105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.932388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] PATH_INFO: `/v3/services/network` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.933333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.934940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:24.950891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:24.952002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:24.952892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:24.953641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 372/768] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:24.956652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.985415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:25.010546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a80cfc-61dd-4136-8314-36a96a43ef40 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.050974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac67bc-119f-4c22-af0d-12561c286592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.051857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac67bc-119f-4c22-af0d-12561c286592 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.052288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac67bc-119f-4c22-af0d-12561c286592 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.052776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac67bc-119f-4c22-af0d-12561c286592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.083911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ebc6b-4630-4aa1-9ddf-6a6f13e6b2d3 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.125384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac67bc-119f-4c22-af0d-12561c286592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.127298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac67bc-119f-4c22-af0d-12561c286592 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:25.127524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.128195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.128483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.128793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.129588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] RBAC: Adding query filter params (interface=public, service_id=399e051737d84bdb9beb61d28a1df4ba, region_id=RegionOne) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:25.129907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.132222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:25.146924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 373/769] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/endpoints?service_id=399e051737d84bdb9beb61d28a1df4ba&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:25.146924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:25.198353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:25.229596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:25.260598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 397/770] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:25.261762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac67bc-119f-4c22-af0d-12561c286592 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:25.263978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eafe35-c891-4dcc-8da9-deb8fb92a7c1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.303138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.304150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.305432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.305913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.306972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] RBAC: Adding query filter params (name=network) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:25.307587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.309221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:25.314572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 374/771] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:25.315813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:25.395412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac67bc-119f-4c22-af0d-12561c286592 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.423746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bc010-da94-425b-bb08-0997b79e9011 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.433690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:25.440682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a9d6a-ea7b-41c2-b113-06b5fcd8837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.441498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a9d6a-ea7b-41c2-b113-06b5fcd8837e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.441784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a9d6a-ea7b-41c2-b113-06b5fcd8837e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.442072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a9d6a-ea7b-41c2-b113-06b5fcd8837e None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.442761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a9d6a-ea7b-41c2-b113-06b5fcd8837e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.444611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a9d6a-ea7b-41c2-b113-06b5fcd8837e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:25.460559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 398/772] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:25 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:25.460559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a9d6a-ea7b-41c2-b113-06b5fcd8837e None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:25.478071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.479322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.479806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.480302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.502897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.503511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:25.506216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:25.536020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:25.537097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:25.538011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:25.538011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:25.538011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:25.538011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:25.541172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 375/773] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:25.550955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:25.607465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a9d6a-ea7b-41c2-b113-06b5fcd8837e None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.649474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.650069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.650351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.650627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.651349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] RBAC: Adding query filter params (type=network) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:25.651654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.653086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:25.662661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 399/774] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:25.670839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:25.729167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02e1b-e699-47a5-8848-7f34ac6161e1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.777858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.779021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.779310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.779876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.780643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:25.781097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.783251 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:25.798850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 376/775] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:25.807395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:25.821753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a3100-1d31-4e39-b5d2-ceddaa479d22 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.832849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:25.848893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:25.936574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.937166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.938138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.940324 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c706ce-64a7-4ae3-9f50-6c7df36395b1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:25.941539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.942352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] RBAC: Adding query filter params (interface=public, service_id=8068aa76e38043dba4cf8ddf649e1899, region_id=RegionOne) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:25.942673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.944033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:25.962657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 400/776] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jun 16 00:17:25 2022] GET /identity/v3/endpoints?service_id=8068aa76e38043dba4cf8ddf649e1899&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:25.968025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:25.983403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:25.984673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:25.984972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:25.985555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:25.994713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:25.996937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:26.011883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:26.012862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:26.013783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 377/777] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:26.018916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:26.112810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:26.134631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bb0f7-3984-4aa1-b493-7a3abab492b9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:26.185240 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:26.185912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:26.186198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:26.186477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:26.187331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:26.187586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:26.189064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:26.200570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7f97-f458-4886-8dff-f7cdd743c2be None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:26.216626 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 378/778] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:26.221995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:26.290550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86184d-cb72-4335-bb01-9dd0607edd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:26.291231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86184d-cb72-4335-bb01-9dd0607edd5f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:26.291514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86184d-cb72-4335-bb01-9dd0607edd5f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:26.291785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86184d-cb72-4335-bb01-9dd0607edd5f None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:26.292474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86184d-cb72-4335-bb01-9dd0607edd5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:26.293880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86184d-cb72-4335-bb01-9dd0607edd5f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:26.312229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 401/779] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:25 2022] GET /identity/v3/services => generated 772 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:26.399089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5998d-bdce-466c-af4a-2b679ce84878 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:26.443322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c11b7d-0e61-4b81-830d-8198ece9d475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:26.443929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c11b7d-0e61-4b81-830d-8198ece9d475 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:26.444212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c11b7d-0e61-4b81-830d-8198ece9d475 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:26.444489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c11b7d-0e61-4b81-830d-8198ece9d475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:26.445422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c11b7d-0e61-4b81-830d-8198ece9d475 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=396cdfa4dbf54e66919bc3e054c54154) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:26.445739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c11b7d-0e61-4b81-830d-8198ece9d475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:26.447208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c11b7d-0e61-4b81-830d-8198ece9d475 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:26.453874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 379/780] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:26 2022] GET /identity/v3/role_assignments?user.id=396cdfa4dbf54e66919bc3e054c54154&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:26.740361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37445-00a5-448e-996b-b42ca2a94b07 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:26.740361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37445-00a5-448e-996b-b42ca2a94b07 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:26.740361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37445-00a5-448e-996b-b42ca2a94b07 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:26.741620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 402/781] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:26.747441 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c02182-bb0a-4f28-a943-971127b543ac None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:26.747963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c02182-bb0a-4f28-a943-971127b543ac None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:26.748471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c02182-bb0a-4f28-a943-971127b543ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:26.754849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:26.766600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-41c02182-bb0a-4f28-a943-971127b543ac None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:26.768048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-41c02182-bb0a-4f28-a943-971127b543ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:26.842621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 380/782] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:26.864925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb37445-00a5-448e-996b-b42ca2a94b07 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:26.865482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658522-a4e8-4b1c-ad9c-50e9f59df147 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:26.865775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658522-a4e8-4b1c-ad9c-50e9f59df147 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:26.866055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658522-a4e8-4b1c-ad9c-50e9f59df147 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:26.868690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 381/783] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:26.873045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00711a7-b811-4714-996d-4a6c419a5e71 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:26.873335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00711a7-b811-4714-996d-4a6c419a5e71 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:26.873618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00711a7-b811-4714-996d-4a6c419a5e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:26.903668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e00711a7-b811-4714-996d-4a6c419a5e71 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:26.904993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e00711a7-b811-4714-996d-4a6c419a5e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:26.954305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:27.003664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 382/784] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:27.038439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00711a7-b811-4714-996d-4a6c419a5e71 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.074568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb37445-00a5-448e-996b-b42ca2a94b07 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.134602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:27.155079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239a8aa-47bd-4b11-ba5f-e9cee07cf67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.156242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239a8aa-47bd-4b11-ba5f-e9cee07cf67d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.156781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239a8aa-47bd-4b11-ba5f-e9cee07cf67d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.157284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239a8aa-47bd-4b11-ba5f-e9cee07cf67d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.175679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00711a7-b811-4714-996d-4a6c419a5e71 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.177254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239a8aa-47bd-4b11-ba5f-e9cee07cf67d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-8239a8aa-47bd-4b11-ba5f-e9cee07cf67d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:27.195908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:27.197869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:27.199597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 403/785] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:27.206910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239a8aa-47bd-4b11-ba5f-e9cee07cf67d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.218689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8818f-f07a-41c8-a48d-48ae6f06a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.219489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8818f-f07a-41c8-a48d-48ae6f06a1ef None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.219773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8818f-f07a-41c8-a48d-48ae6f06a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.220053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8818f-f07a-41c8-a48d-48ae6f06a1ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.249604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8818f-f07a-41c8-a48d-48ae6f06a1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-b9f8818f-f07a-41c8-a48d-48ae6f06a1ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:27.257311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:27.260020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:27.262075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 383/786] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:27.269498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8818f-f07a-41c8-a48d-48ae6f06a1ef None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.376760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:27.385890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239a8aa-47bd-4b11-ba5f-e9cee07cf67d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.449696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8818f-f07a-41c8-a48d-48ae6f06a1ef None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.474408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:27.485821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.486267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.486507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.488673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.488846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.489017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.489191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.489191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.489588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:27.489588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:27.489880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:27.489880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:27.490155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:27.490299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:27.506374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 404/787] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:27.508883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 384/788] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:27.512588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.514278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.662239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95202f7e-3725-4b32-81ef-611a66e42137 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.729641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.730438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.730724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.731000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.742308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeed081-d3b8-4639-9833-4dfdf9783cd2 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.753725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:27.755189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:27.784828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:27.785772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:27.786694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:27.786694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:27.786694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:27.786694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:27.786694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 405/789] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:27.792213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.823499 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.824362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.824677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.824958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.834309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:27.857053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:27.859053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:27.890783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:27.891844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:27.892789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:27.892789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:27.892789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:27.892789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:27.892789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 385/790] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:27.904462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.923795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec04d6-d9a3-4771-a459-78c43c55f6e5 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.975410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.976081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.977016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.977307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.978039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:27.978351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:27.979721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:27.997903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.007313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 406/791] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:28.013413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.127589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265501e-c06a-45bf-9f8e-d9cc5ab7e641 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.137796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.173608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da021f-6f9e-4725-bd92-3d16e40d2eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.174589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da021f-6f9e-4725-bd92-3d16e40d2eec None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.174873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da021f-6f9e-4725-bd92-3d16e40d2eec None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.175154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da021f-6f9e-4725-bd92-3d16e40d2eec None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.176352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da021f-6f9e-4725-bd92-3d16e40d2eec None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:28.176716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da021f-6f9e-4725-bd92-3d16e40d2eec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.178418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da021f-6f9e-4725-bd92-3d16e40d2eec None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.179334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecb644-8107-4c5e-8b10-f2cd953851ea None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.191734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 386/792] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:28.193387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9b83-7037-46e4-a7c3-db57f4593a64 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.193680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9b83-7037-46e4-a7c3-db57f4593a64 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.194117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9b83-7037-46e4-a7c3-db57f4593a64 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.196646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 387/793] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:28.197478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9b83-7037-46e4-a7c3-db57f4593a64 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.219361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bae6ac-8d19-4587-a9a1-4eff0fbc3168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.220470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae6ac-8d19-4587-a9a1-4eff0fbc3168 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.221170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae6ac-8d19-4587-a9a1-4eff0fbc3168 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.221554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae6ac-8d19-4587-a9a1-4eff0fbc3168 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.231602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bae6ac-8d19-4587-a9a1-4eff0fbc3168 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.233625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bae6ac-8d19-4587-a9a1-4eff0fbc3168 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-48bae6ac-8d19-4587-a9a1-4eff0fbc3168 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:28.251261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:28.252247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:28.253234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:28.253777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 407/794] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:28.255310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382915-389d-4569-a0c4-574dfa9da776 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.255671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382915-389d-4569-a0c4-574dfa9da776 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.256024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382915-389d-4569-a0c4-574dfa9da776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.278207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-8a382915-389d-4569-a0c4-574dfa9da776 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:28.280027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8a382915-389d-4569-a0c4-574dfa9da776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:28.312811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.317598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9b83-7037-46e4-a7c3-db57f4593a64 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.353600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.354508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.354798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.355098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.356694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 408/795] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:28.357796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a382915-389d-4569-a0c4-574dfa9da776 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.363382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.365382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:28.379956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:28.380984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:28.381986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 388/796] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:28.393465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.424447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.473237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a382915-389d-4569-a0c4-574dfa9da776 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.510479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547224f-e788-44d7-ab03-9c0b1f363245 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.513636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.514431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.514854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.515236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.516236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:28.516705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.518237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.536223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 409/797] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:28.537238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.552495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.553776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.554171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.555076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.556638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:28.557058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.559125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.588964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 389/798] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:28.593165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.613735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.654456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88608713-0c6f-43a6-afc9-b9f7f9ce8e5d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.672954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.691472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.692652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.693137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.693596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.694442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.696552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.702139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfdf90-775e-4a34-88b6-b84d83e4312d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:28.711696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:28.713374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:17:28.714245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:28.716037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 410/799] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:28.716681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.751871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.752961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.753317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.754020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.768421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.770135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:28.795755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:28.796762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:28.797677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:28.797677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:28.797677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:28.797677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:28.798642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 390/800] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:28.798791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.830683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c606cb-8b08-4435-96cf-0de614f84527 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.868107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c553be5-c8e6-48ae-9ff3-826f6a33fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.868814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553be5-c8e6-48ae-9ff3-826f6a33fe94 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.869104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553be5-c8e6-48ae-9ff3-826f6a33fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.869382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553be5-c8e6-48ae-9ff3-826f6a33fe94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.925195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.946318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c553be5-c8e6-48ae-9ff3-826f6a33fe94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.947794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c553be5-c8e6-48ae-9ff3-826f6a33fe94 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.992712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.031699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e7ed9-308d-4328-ae58-671f8d3be50c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.065832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 411/801] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:29.065832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772eab4-ea18-4497-9a7e-a818c085fe88 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.065832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772eab4-ea18-4497-9a7e-a818c085fe88 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.065832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772eab4-ea18-4497-9a7e-a818c085fe88 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.065832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 412/802] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:29.065832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772eab4-ea18-4497-9a7e-a818c085fe88 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.077004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.077680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.078165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.078271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.078924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:29.079485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.080611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.090311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 391/803] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:29.090311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.155926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.175538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772eab4-ea18-4497-9a7e-a818c085fe88 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.184637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7b8f9-53e4-4c4f-9839-e456b02ce460 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.208580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1147f0-b484-488f-834b-15f4b6493bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.209242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1147f0-b484-488f-834b-15f4b6493bfe None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.209531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1147f0-b484-488f-834b-15f4b6493bfe None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.209812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1147f0-b484-488f-834b-15f4b6493bfe None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.210558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1147f0-b484-488f-834b-15f4b6493bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:29.211421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1147f0-b484-488f-834b-15f4b6493bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.212810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1147f0-b484-488f-834b-15f4b6493bfe None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.213927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.218705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.221382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.221382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.221382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.223815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 413/804] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:29.225128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4972ddc-7afc-4b27-83ba-36878bc160e4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.225391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4972ddc-7afc-4b27-83ba-36878bc160e4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.225668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4972ddc-7afc-4b27-83ba-36878bc160e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.230605 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.232037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.241426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e4972ddc-7afc-4b27-83ba-36878bc160e4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:29.243132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e4972ddc-7afc-4b27-83ba-36878bc160e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:29.249655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:29.250927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:29.253247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:29.253247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:29.253247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:29.253247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:29.253247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 392/805] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:29.253247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.313629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 414/806] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:29.315225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050cb3e-1845-4bad-89b4-e196b2c568ea None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.315673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050cb3e-1845-4bad-89b4-e196b2c568ea None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.316107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050cb3e-1845-4bad-89b4-e196b2c568ea None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.317860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 415/807] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:29.318772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050cb3e-1845-4bad-89b4-e196b2c568ea None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.354146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3730b2-07a2-465a-9212-6e8fad0f7c38 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.388111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.388808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.389098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.389389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.390093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:29.390406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.391773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.394289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.396619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 393/808] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:29.397424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.427261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050cb3e-1845-4bad-89b4-e196b2c568ea None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.443997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.461780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bf4eb-67b8-40e8-b76f-19a5a51df950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.462570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bf4eb-67b8-40e8-b76f-19a5a51df950 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.462861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bf4eb-67b8-40e8-b76f-19a5a51df950 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.463143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bf4eb-67b8-40e8-b76f-19a5a51df950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.469779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bf4eb-67b8-40e8-b76f-19a5a51df950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.471218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bf4eb-67b8-40e8-b76f-19a5a51df950 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-f42bf4eb-67b8-40e8-b76f-19a5a51df950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:29.485294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:29.486200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:29.487122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 416/809] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:29.487798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c16b-d628-4c4c-8da9-e77ed03be9f6 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.488052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c16b-d628-4c4c-8da9-e77ed03be9f6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.488327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c16b-d628-4c4c-8da9-e77ed03be9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.497943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba04f92-f719-411b-b8f1-6c3d8f4f4872 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.504782 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-52c1c16b-d628-4c4c-8da9-e77ed03be9f6 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:29.505964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-52c1c16b-d628-4c4c-8da9-e77ed03be9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:29.530753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.531413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.531706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.531985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.532741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:29.533052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.534411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.546183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 394/810] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:29.546970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.573025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 417/811] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:29.573842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c16b-d628-4c4c-8da9-e77ed03be9f6 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.637160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.651313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aa5fc-ed88-4d34-afa8-18affc5a449a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.659901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.677191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c16b-d628-4c4c-8da9-e77ed03be9f6 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.685469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.686202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.686496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.686884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] PATH_INFO: `/v3/services/network` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.687690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.689193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:29.702702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:29.703818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:29.704749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 395/812] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:29.705751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.718474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524a5f2-3e9f-4f18-85f2-e892951401e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.719146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524a5f2-3e9f-4f18-85f2-e892951401e3 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.719435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524a5f2-3e9f-4f18-85f2-e892951401e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.719715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524a5f2-3e9f-4f18-85f2-e892951401e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.720513 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524a5f2-3e9f-4f18-85f2-e892951401e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.721897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524a5f2-3e9f-4f18-85f2-e892951401e3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.801818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a234a94-1e7b-4acb-8250-690f5c362f4c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.813272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 418/813] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:17:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:29.814170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524a5f2-3e9f-4f18-85f2-e892951401e3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.855448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.856138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.856423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.856811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.857606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:29.857922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.859426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.879184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 396/814] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:29.879184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.920064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.935355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524a5f2-3e9f-4f18-85f2-e892951401e3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.954175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.978153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.979007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.979291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.979563 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.988574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.990039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:30.004931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:30.005840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:30.006742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 419/815] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:30.007401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:30.010190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cbbd4-c46b-47b7-8178-229b67da5705 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.060838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea63f96-c475-4507-b519-dfb3fae838b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.061800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea63f96-c475-4507-b519-dfb3fae838b5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.062259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea63f96-c475-4507-b519-dfb3fae838b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.062688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea63f96-c475-4507-b519-dfb3fae838b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.074483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea63f96-c475-4507-b519-dfb3fae838b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-aea63f96-c475-4507-b519-dfb3fae838b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:30.082253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:30.083244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:30.084096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 397/816] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:30.085136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea63f96-c475-4507-b519-dfb3fae838b5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:30.122510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9140d5-34c2-4e7a-b8dd-7d93034510e7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.157538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.158208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.158491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.158767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.159474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] RBAC: Adding query filter params (name=network) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:30.159780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:30.161167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.166298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 420/817] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:30.167091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:30.183669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:30.195876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea63f96-c475-4507-b519-dfb3fae838b5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.212261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:30.232150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45b9cc-bb51-42c0-bd14-f2cbea185ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.233070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45b9cc-bb51-42c0-bd14-f2cbea185ea5 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.233356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45b9cc-bb51-42c0-bd14-f2cbea185ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.233634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45b9cc-bb51-42c0-bd14-f2cbea185ea5 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/645bbe41756749b98232d0e2eb2e678a/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.235604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45b9cc-bb51-42c0-bd14-f2cbea185ea5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=645bbe41756749b98232d0e2eb2e678a, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:30.237070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45b9cc-bb51-42c0-bd14-f2cbea185ea5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.274179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f12a1a-4414-4b03-bb12-79557b032422 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.306485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 398/818] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:30 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/645bbe41756749b98232d0e2eb2e678a/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:30.307714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45b9cc-bb51-42c0-bd14-f2cbea185ea5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:30.325663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.326461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.326760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.327046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.328654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:30.328980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:30.331243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.368157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 421/819] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:30.369190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:30.457950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45b9cc-bb51-42c0-bd14-f2cbea185ea5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.528456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.529199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.529631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.529920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.530821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:30.531345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:30.532494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.545041 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:30.547112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 399/820] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:30.550097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:30.555258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d375576-6bff-48d5-a1c5-5262fbf3ebe6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.563502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:30.642504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.644402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.646372 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.647701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.650367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] RBAC: Adding query filter params (type=network) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:30.651965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:30.656847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.677648 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 422/821] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:30.678981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:30.681403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daec3f-bdbb-4844-8044-665276f56fdf None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.752319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.753369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.753947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.754646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.755642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=dea47e0ec737422c8f679e21174f8cab) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:30.756150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:30.762367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.777963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 400/822] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:30 2022] GET /identity/v3/role_assignments?user.id=dea47e0ec737422c8f679e21174f8cab&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:30.779199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:30.857151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a200d9-3a58-4ba7-bf87-90e3bb5be090 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.879898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:30.907844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a0463-cf54-4c44-b493-9b893f1a2740 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:30.914015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.914859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.915219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.915572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.947543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:30.949616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.952765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864aa37d-690e-4eef-a9d8-03b8ece6e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:30.953834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aa37d-690e-4eef-a9d8-03b8ece6e3ba None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:30.954292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aa37d-690e-4eef-a9d8-03b8ece6e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:30.954742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aa37d-690e-4eef-a9d8-03b8ece6e3ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:30.955707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864aa37d-690e-4eef-a9d8-03b8ece6e3ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:30.957349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864aa37d-690e-4eef-a9d8-03b8ece6e3ba None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:30.985535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:30.987886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:30.989070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:30.989070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:30.989070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:30.989070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:30.990053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 423/823] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:30.996663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:31.018168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:31.048166 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 401/824] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:17:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:31.124455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092d41f-d857-4bcb-8de5-1ad8f2741206 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:31.170287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:31.170980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:31.171302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:31.171631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:31.172422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:31.172773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:31.174440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:31.188399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 424/825] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:31.193887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864aa37d-690e-4eef-a9d8-03b8ece6e3ba None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:31.314042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864aa37d-690e-4eef-a9d8-03b8ece6e3ba None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:31.345280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:31.351822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:31.352823 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:31.353205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:31.353579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:31.362631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:31.366962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:31.381237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:31.382442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:31.383619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:31.385522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 402/826] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:31.391132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:31.471493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:31.508254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82e27d-13e2-452e-8883-2363908ea99a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:31.547379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:31.548049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:31.548330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:31.548881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:31.549708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:31.550018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:31.551435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:31.570368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 425/827] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jun 16 00:17:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:31.581655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:31.695338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d8143-3c76-4680-bb5c-c8b671d02430 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:31.735057 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:31.735861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:31.736149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:31.736434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:31.747808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:31.749327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:31.767223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:31.768139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:31.769083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:31.769083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:31.769083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:31.769083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:31.769083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 403/828] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:31.775874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:31.884513 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d75d3-fc50-4ca3-bea5-50a0bea4ad6c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:31.915506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:31.921491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19769ca6-3d71-43b2-9af6-b91b11a0c1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:31.922163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19769ca6-3d71-43b2-9af6-b91b11a0c1a1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:31.922495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19769ca6-3d71-43b2-9af6-b91b11a0c1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:31.922782 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19769ca6-3d71-43b2-9af6-b91b11a0c1a1 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:31.923776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19769ca6-3d71-43b2-9af6-b91b11a0c1a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:31.924091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19769ca6-3d71-43b2-9af6-b91b11a0c1a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:31.925625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19769ca6-3d71-43b2-9af6-b91b11a0c1a1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:31.937489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 426/829] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:31.944955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:31.991328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.058211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69ed24-63ab-42de-becd-b994a4e48104 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.080965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc544a1-7c47-43ef-9793-0ad67fa206fc None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.081500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc544a1-7c47-43ef-9793-0ad67fa206fc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.081500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc544a1-7c47-43ef-9793-0ad67fa206fc None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.083079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 427/830] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:32.087935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b956-d85a-4810-97cf-16d32a877bfc None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.088227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b956-d85a-4810-97cf-16d32a877bfc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.088507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b956-d85a-4810-97cf-16d32a877bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.093305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.094392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.095591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.095883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.104255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.105738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.106292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5571b956-d85a-4810-97cf-16d32a877bfc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:32.107548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5571b956-d85a-4810-97cf-16d32a877bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:32.124569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:32.125571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:32.126479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:32.126479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:32.126479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:32.126479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:32.126479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:32.126479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:32.126479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:32.126479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:32.129509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 404/831] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:32.135275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.184615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 428/832] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:32.202776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571b956-d85a-4810-97cf-16d32a877bfc None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.259139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca849ba-815c-4a98-b6f8-4ebc88fab293 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.272888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.304857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.305529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.305814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.306092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.306886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:32.307200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.308660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.325568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 405/833] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:32.326952 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.331985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571b956-d85a-4810-97cf-16d32a877bfc None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.354251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.369811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.374143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.374143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.374143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.374143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.374143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:32.387464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:32.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:32.389526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:32.397971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 429/834] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:32.399616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.431791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff221-0fbc-4969-9c8d-523e1ce532bb None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.468296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f332a-d119-43d0-951f-5d7f918ad9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.469234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f332a-d119-43d0-951f-5d7f918ad9d1 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.469520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f332a-d119-43d0-951f-5d7f918ad9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.469960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f332a-d119-43d0-951f-5d7f918ad9d1 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/396cdfa4dbf54e66919bc3e054c54154/roles/78418cd7aaaf481e94926d572280f81e` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.489562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f332a-d119-43d0-951f-5d7f918ad9d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=396cdfa4dbf54e66919bc3e054c54154, role_id=78418cd7aaaf481e94926d572280f81e)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.491388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f332a-d119-43d0-951f-5d7f918ad9d1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.507633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f575e6-7f77-4e0c-bb97-83f849364930 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.549509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.550266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.550496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.550765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.551535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:32.551785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.553241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.558081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 430/835] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:32.569232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.576668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 406/836] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:32 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/396cdfa4dbf54e66919bc3e054c54154/roles/78418cd7aaaf481e94926d572280f81e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:32.639595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.672838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe829-847d-42a1-93fb-aee212b25b38 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.673647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe829-847d-42a1-93fb-aee212b25b38 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.673647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe829-847d-42a1-93fb-aee212b25b38 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.675353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 407/837] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:32.680461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4046d-934c-4e03-8733-584aeb85df6b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.680781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4046d-934c-4e03-8733-584aeb85df6b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.681066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4046d-934c-4e03-8733-584aeb85df6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.699839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-94d4046d-934c-4e03-8733-584aeb85df6b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:32.701178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-94d4046d-934c-4e03-8733-584aeb85df6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:32.712901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.714798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa8298-202d-43b8-9ea6-8473d4bf620a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.759921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.759921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.760363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.760603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.761316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:32.761630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.763004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.772556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 431/838] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:32.789492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 408/839] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:32.816016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.937600 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c088682-2edf-4649-a530-e3c39c3109d4 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.951943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.976722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c8682-5301-4053-9ba8-c20dcab5888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.978062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c8682-5301-4053-9ba8-c20dcab5888f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.978539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c8682-5301-4053-9ba8-c20dcab5888f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.978819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c8682-5301-4053-9ba8-c20dcab5888f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.991067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c8682-5301-4053-9ba8-c20dcab5888f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-723c8682-5301-4053-9ba8-c20dcab5888f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:32.999239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:33.000191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:33.001076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 432/840] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:33.009183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4046d-934c-4e03-8733-584aeb85df6b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.112207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4046d-934c-4e03-8733-584aeb85df6b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.152760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.153440 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.153730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.154013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.154772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:33.155083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.156453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.165862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 409/841] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:33.171588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c8682-5301-4053-9ba8-c20dcab5888f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.287350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c8682-5301-4053-9ba8-c20dcab5888f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.323714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccbcac-0cbb-47f0-8c7e-2dabc85312be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.324533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccbcac-0cbb-47f0-8c7e-2dabc85312be None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.324824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccbcac-0cbb-47f0-8c7e-2dabc85312be None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.325104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccbcac-0cbb-47f0-8c7e-2dabc85312be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.336655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccbcac-0cbb-47f0-8c7e-2dabc85312be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.338105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccbcac-0cbb-47f0-8c7e-2dabc85312be None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-13ccbcac-0cbb-47f0-8c7e-2dabc85312be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:33.356099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:33.357336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:33.358427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:33.358427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:33.358427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:33.358427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:33.358427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 433/842] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:33.365624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.378851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.462781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e58c0f-a93e-4ac9-b62e-9cb9864164d6 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.463265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e58c0f-a93e-4ac9-b62e-9cb9864164d6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.463555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e58c0f-a93e-4ac9-b62e-9cb9864164d6 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.465583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 434/843] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:33.471334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e71b0-44c7-4034-9e31-f17c28b93aa6 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.471623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e71b0-44c7-4034-9e31-f17c28b93aa6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.471912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e71b0-44c7-4034-9e31-f17c28b93aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.476172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3176-1baf-4d8d-ae15-f32e8dbca478 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.495044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d78e71b0-44c7-4034-9e31-f17c28b93aa6 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:33.497029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d78e71b0-44c7-4034-9e31-f17c28b93aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:33.516172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.516868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.517150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.517424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.518409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:33.518716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.519852 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.521139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.536990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 410/844] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:33.542858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.593954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 435/845] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:33.617829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e71b0-44c7-4034-9e31-f17c28b93aa6 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.639069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.688620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b325b-064c-4344-9abe-23bd0059bd73 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.742486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.743323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e71b0-44c7-4034-9e31-f17c28b93aa6 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.743771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.744200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.744655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.756139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.758651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:33.779130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:33.780132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:17:33.781512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:33.782079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 411/846] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:33.788638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.800378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.801892 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.802536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.802831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.803787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.805336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:33.827711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:33.828760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:33.829632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:33.833876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 436/847] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:33.836825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.858144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.925429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20722c7-fe03-4540-840b-9c304fde8b9d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.949572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a22ff-adb3-4174-8dbf-340f38e8e37a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.957396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.974231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.974915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.975203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.975486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.977133 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:33.977632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.980001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.998309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.998614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 412/848] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:33.999602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.000116 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.000627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.001544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:34.002057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:34.003695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:34.004251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:34.015788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 437/849] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:34.016387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:34.120178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:34.131336 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3a99f-c62a-431e-9dba-bb2078e2dae6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:34.145711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc6588-94e0-48a2-a813-c7efddd6a204 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:34.167571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a07e6a-4fa4-411d-acb8-f44c51ef34ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:34.168219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a07e6a-4fa4-411d-acb8-f44c51ef34ce None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.168502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a07e6a-4fa4-411d-acb8-f44c51ef34ce None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.168817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a07e6a-4fa4-411d-acb8-f44c51ef34ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.189992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:34.190662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.190931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.191206 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.191933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:34.192204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:34.193705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:34.204550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 438/850] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:34.236695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:34.256067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a07e6a-4fa4-411d-acb8-f44c51ef34ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:34.258105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a07e6a-4fa4-411d-acb8-f44c51ef34ce None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:34.371193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 413/851] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:17:34.376598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:34.492786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c57ec-a0ef-4313-8502-6daecda5622d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:34.533171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:34.533978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.534264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.534544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.546137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:34.547603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:34.555143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:34.566556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:34.573052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:34.573968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:34.573968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:34.573968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:34.573968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:34.573968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 439/852] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:34.575832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a07e6a-4fa4-411d-acb8-f44c51ef34ce None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:34.667802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:34.687505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a07e6a-4fa4-411d-acb8-f44c51ef34ce None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:34.727202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cd4d0-f0a2-4ff0-a70d-cc6d409476d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:34.727942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cd4d0-f0a2-4ff0-a70d-cc6d409476d4 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.728161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cd4d0-f0a2-4ff0-a70d-cc6d409476d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.728442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cd4d0-f0a2-4ff0-a70d-cc6d409476d4 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.729269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cd4d0-f0a2-4ff0-a70d-cc6d409476d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:34.729580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cd4d0-f0a2-4ff0-a70d-cc6d409476d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:34.730985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cd4d0-f0a2-4ff0-a70d-cc6d409476d4 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:34.743371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 414/853] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:34.748938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:34.833772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a4ec1-749f-4b49-9b21-366b8e3361f3 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.834090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a4ec1-749f-4b49-9b21-366b8e3361f3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.834336 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a4ec1-749f-4b49-9b21-366b8e3361f3 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.835880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 415/854] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:34.841197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59269-63f2-4637-8006-723e7a5d1387 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.841770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59269-63f2-4637-8006-723e7a5d1387 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.842262 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59269-63f2-4637-8006-723e7a5d1387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.860081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6cd59269-63f2-4637-8006-723e7a5d1387 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:34.861658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6cd59269-63f2-4637-8006-723e7a5d1387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:34.868724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97444d5-4b69-41bd-8288-a391ae0d5e96 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:34.914695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:34.915498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.916163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.916457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.925625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:34.927763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:34.944187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:34.945422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:34.946862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 440/856] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:34.947559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 416/856] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 16 00:17:34.951097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:34.966426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59269-63f2-4637-8006-723e7a5d1387 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:35.033693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:35.045274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:35.075407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59269-63f2-4637-8006-723e7a5d1387 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:35.077012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370c085-5f71-4c28-82c9-ed04bfccd2cb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:35.111063 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2c637-461a-46a6-a48e-b21467e57618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:35.111861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2c637-461a-46a6-a48e-b21467e57618 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.112151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2c637-461a-46a6-a48e-b21467e57618 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.112434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2c637-461a-46a6-a48e-b21467e57618 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.113111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:35.114055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.114711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.115411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.116344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:35.116884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:35.119133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:35.123407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2c637-461a-46a6-a48e-b21467e57618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3ae2c637-461a-46a6-a48e-b21467e57618 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:35.131090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:35.132234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:35.133367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:35.139291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 417/857] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:35.139291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e045-0e06-4510-a007-cdff16fe380f None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.139291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e045-0e06-4510-a007-cdff16fe380f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.139291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e045-0e06-4510-a007-cdff16fe380f None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.139772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 441/858] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:35.140571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 418/859] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:35.140924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:35.141270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e045-0e06-4510-a007-cdff16fe380f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:35.247118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e045-0e06-4510-a007-cdff16fe380f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:35.248867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a42385-af08-421f-a67d-e08347a91942 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:35.270807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:35.276882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:35.281653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92eae1-efce-4d0e-a90c-ca92c171166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:35.282311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92eae1-efce-4d0e-a90c-ca92c171166c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.282599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92eae1-efce-4d0e-a90c-ca92c171166c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.282886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92eae1-efce-4d0e-a90c-ca92c171166c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.283713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92eae1-efce-4d0e-a90c-ca92c171166c None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=645bbe41756749b98232d0e2eb2e678a) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:35.284036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92eae1-efce-4d0e-a90c-ca92c171166c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:35.285490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92eae1-efce-4d0e-a90c-ca92c171166c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:35.291880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 419/860] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:35 2022] GET /identity/v3/role_assignments?user.id=645bbe41756749b98232d0e2eb2e678a&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:35.292438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:35.293165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cac6a7-c52d-4dca-84f6-c242ec83680d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.293461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cac6a7-c52d-4dca-84f6-c242ec83680d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.293688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.293788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cac6a7-c52d-4dca-84f6-c242ec83680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.294215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.294656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.295596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:35.295909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:35.297325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:35.316304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 442/861] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:35.318799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:35.324604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-47cac6a7-c52d-4dca-84f6-c242ec83680d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:35.326082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-47cac6a7-c52d-4dca-84f6-c242ec83680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:35.411169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 420/862] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:35.434911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cac6a7-c52d-4dca-84f6-c242ec83680d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:35.450100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8366a-5342-4e53-849b-f6c9c06e6622 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:35.502997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:35.503812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.504112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.504385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.515748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:35.517276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:35.534974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:35.535906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:35.537711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:35.537711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:35.537711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:35.537711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:35.537711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 443/863] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:35.542285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:35.558952 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cac6a7-c52d-4dca-84f6-c242ec83680d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:35.586592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:35.598436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961361-b46f-4c39-8d06-91147def77df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:35.599114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961361-b46f-4c39-8d06-91147def77df None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.599399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961361-b46f-4c39-8d06-91147def77df None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.600665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961361-b46f-4c39-8d06-91147def77df None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.600665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99961361-b46f-4c39-8d06-91147def77df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:35.602340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99961361-b46f-4c39-8d06-91147def77df None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:35.602622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:35.676183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a458067-84b9-4d2d-8e17-141a58d687a4 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:35.677980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 421/864] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:35 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:35.729825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:35.730844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.731318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.731781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.732769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:35.733278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:35.734945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:35.749655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 444/865] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:35.759163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961361-b46f-4c39-8d06-91147def77df None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:35.879020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961361-b46f-4c39-8d06-91147def77df None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:35.914337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:35.915165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:35.915452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:35.915738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:35.924199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:35.925678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:35.941349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:35.942362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:35.943330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 422/866] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:35.950094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:36.046807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:36.059920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9b842-ce5a-4803-989e-c7fa0e8665eb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:36.096505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeed5e7-2717-4aa9-baad-7a5dbaf0e21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:36.097208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeed5e7-2717-4aa9-baad-7a5dbaf0e21b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.097499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeed5e7-2717-4aa9-baad-7a5dbaf0e21b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.097769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeed5e7-2717-4aa9-baad-7a5dbaf0e21b None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.098626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeed5e7-2717-4aa9-baad-7a5dbaf0e21b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:36.098939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeed5e7-2717-4aa9-baad-7a5dbaf0e21b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:36.100375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeed5e7-2717-4aa9-baad-7a5dbaf0e21b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:36.119806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 445/867] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jun 16 00:17:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:36.125370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:36.172492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:36.239991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2205ab-ae1a-495d-95e5-e715a7f7dacf None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:36.278046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edd040-b512-4695-914e-e7731ca2b41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:36.278695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edd040-b512-4695-914e-e7731ca2b41a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.278981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edd040-b512-4695-914e-e7731ca2b41a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.279263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edd040-b512-4695-914e-e7731ca2b41a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.352492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edd040-b512-4695-914e-e7731ca2b41a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:36.353936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edd040-b512-4695-914e-e7731ca2b41a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:36.405643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b76123-ca50-4763-8243-fb068d3dc015 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.406188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b76123-ca50-4763-8243-fb068d3dc015 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.406188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b76123-ca50-4763-8243-fb068d3dc015 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.407760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 446/868] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:36.408203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:36.412814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9acb9-8478-4e45-a068-a53fa7859ff3 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.413099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9acb9-8478-4e45-a068-a53fa7859ff3 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.413375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9acb9-8478-4e45-a068-a53fa7859ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.433409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-61d9acb9-8478-4e45-a068-a53fa7859ff3 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:36.434865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-61d9acb9-8478-4e45-a068-a53fa7859ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:36.473713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 423/869] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:36 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:36.479467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edd040-b512-4695-914e-e7731ca2b41a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:36.520734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 447/870] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:36.543553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9acb9-8478-4e45-a068-a53fa7859ff3 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:36.599386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edd040-b512-4695-914e-e7731ca2b41a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:36.634931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:36.636060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.636568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.637044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.647473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:36.649735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:36.651743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:36.657193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:36.667863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:36.670014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:36.670938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:36.670938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:36.670938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:36.670938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:36.672064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 424/871] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:36.674682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9acb9-8478-4e45-a068-a53fa7859ff3 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:36.682112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:36.716753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ccc5c-1870-4020-8975-54fe8bdd5c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:36.717422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ccc5c-1870-4020-8975-54fe8bdd5c92 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.717705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ccc5c-1870-4020-8975-54fe8bdd5c92 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.717980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ccc5c-1870-4020-8975-54fe8bdd5c92 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.718708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ccc5c-1870-4020-8975-54fe8bdd5c92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:36.720059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ccc5c-1870-4020-8975-54fe8bdd5c92 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:36.789236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bc483-4eda-4456-a8d7-f03f1f631882 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:36.800447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 448/872] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:36 2022] POST /identity/v3/services => generated 269 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:17:36.840586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:36.841281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.841574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.841910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.842762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:36.843088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:36.844474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:36.858620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 425/873] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:36.864217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ccc5c-1870-4020-8975-54fe8bdd5c92 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:36.980587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ccc5c-1870-4020-8975-54fe8bdd5c92 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.016994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.017802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.018080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.018353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.021542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:37.026167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.027606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:37.042413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:37.043375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:37.044302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 449/874] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:37.049908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.110447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:37.164325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304c45d-abd6-43e1-8aaf-632cda2c9355 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.200230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c105eef-be70-4976-9fff-39578e235f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.201163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c105eef-be70-4976-9fff-39578e235f49 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.201595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c105eef-be70-4976-9fff-39578e235f49 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.202018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c105eef-be70-4976-9fff-39578e235f49 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.203010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c105eef-be70-4976-9fff-39578e235f49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:37.203493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c105eef-be70-4976-9fff-39578e235f49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.205216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c105eef-be70-4976-9fff-39578e235f49 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.218272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 426/875] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:37.224060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.335186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc31fb-2071-4368-93ad-f39402cf6f97 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.371711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68601c56-5317-43ba-ab35-1807210950c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.372650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68601c56-5317-43ba-ab35-1807210950c3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.373083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68601c56-5317-43ba-ab35-1807210950c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.373530 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68601c56-5317-43ba-ab35-1807210950c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.374513 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68601c56-5317-43ba-ab35-1807210950c3 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=396cdfa4dbf54e66919bc3e054c54154) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:37.374991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68601c56-5317-43ba-ab35-1807210950c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.376730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68601c56-5317-43ba-ab35-1807210950c3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.383637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 450/876] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/role_assignments?user.id=396cdfa4dbf54e66919bc3e054c54154&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:37.666047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f499ea9-6f9f-41ec-8978-da3ad645b474 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.666388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f499ea9-6f9f-41ec-8978-da3ad645b474 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.666615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f499ea9-6f9f-41ec-8978-da3ad645b474 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.668205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 427/877] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:37.673975 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb6e66-847c-45e8-9336-cb095573d669 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.674420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb6e66-847c-45e8-9336-cb095573d669 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.674516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb6e66-847c-45e8-9336-cb095573d669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.691797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2cfb6e66-847c-45e8-9336-cb095573d669 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:37.693009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2cfb6e66-847c-45e8-9336-cb095573d669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:37.751533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:37.763785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 451/878] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:37.786668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f499ea9-6f9f-41ec-8978-da3ad645b474 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.901721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f499ea9-6f9f-41ec-8978-da3ad645b474 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.918454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:37.940301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.941209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.941480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.941732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.942418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.943845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:37.960573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:37.963328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:37.967807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 428/879] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:37.968862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.969598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a931b-8344-4a7f-8517-fbad13025b90 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.969888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a931b-8344-4a7f-8517-fbad13025b90 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.970166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a931b-8344-4a7f-8517-fbad13025b90 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.971658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 452/880] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:37 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:37.977493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392923-c52c-4755-a598-5a9c03d06da3 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.977780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392923-c52c-4755-a598-5a9c03d06da3 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.978057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392923-c52c-4755-a598-5a9c03d06da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.995497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-74392923-c52c-4755-a598-5a9c03d06da3 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:37.996493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-74392923-c52c-4755-a598-5a9c03d06da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:38.068778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 453/881] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:38.087445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74392923-c52c-4755-a598-5a9c03d06da3 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.100280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ad37-2ca5-496f-b093-30d5af0267a4 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.143554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.144223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.144510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.144816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.145524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:38.145832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.147182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.152175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 429/882] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:38.158472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.223564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:38.230724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74392923-c52c-4755-a598-5a9c03d06da3 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.257883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:38.276872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.277942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.278413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.278901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.280263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.282276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.293812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997167-4537-474c-8b4c-873d9ca56f8f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:38.300165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:38.301209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:38.302101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:38.306095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 454/883] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:38.311428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.334785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.336070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.336431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.337140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.337928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:38.338343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.339771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.348586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 430/884] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:38.435032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507cdbb-1f15-4245-b84b-7d15ee724df2 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.473061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.474051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.474549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.475012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.475979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:38.477086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.479623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.487448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 455/885] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:38.493458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.610861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a28e1-e147-49a6-97ec-3ca702769a06 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.622632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:38.651351 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.652029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.652317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.652619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.653337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:38.653643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.655004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.660493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 431/886] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:38.666356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.718385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:38.771179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f718-2739-4777-9565-36419bb23d30 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.806126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5284c4-e993-4ad3-89c2-555051af10c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.806785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5284c4-e993-4ad3-89c2-555051af10c1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.807072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5284c4-e993-4ad3-89c2-555051af10c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.807359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5284c4-e993-4ad3-89c2-555051af10c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.808140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5284c4-e993-4ad3-89c2-555051af10c1 None admin] RBAC: Adding query filter params (interface=public, service_id=fc006b90af9746c08846aa10ce79eb93, region_id=RegionOne) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:38.808454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5284c4-e993-4ad3-89c2-555051af10c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.809847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5284c4-e993-4ad3-89c2-555051af10c1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.815290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 456/887] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/endpoints?service_id=fc006b90af9746c08846aa10ce79eb93&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:38.821161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.928413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c86092-8040-41f0-8fc7-584fbbfd48e7 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.964476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b7aea-6fb3-4938-983f-f3e1f94e4c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.965180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7aea-6fb3-4938-983f-f3e1f94e4c3a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.965465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7aea-6fb3-4938-983f-f3e1f94e4c3a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.965743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7aea-6fb3-4938-983f-f3e1f94e4c3a None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.966432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b7aea-6fb3-4938-983f-f3e1f94e4c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.967788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b7aea-6fb3-4938-983f-f3e1f94e4c3a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.974514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 432/888] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:17:39.053085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4595a-eb6c-4844-92bb-1d3a50c952b2 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.053411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4595a-eb6c-4844-92bb-1d3a50c952b2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.053660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4595a-eb6c-4844-92bb-1d3a50c952b2 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.055921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 457/889] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:39.061652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c08d-5d09-486b-845b-c1fad568ba30 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.061947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c08d-5d09-486b-845b-c1fad568ba30 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.062233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c08d-5d09-486b-845b-c1fad568ba30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.080883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8fd7c08d-5d09-486b-845b-c1fad568ba30 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:39.082101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8fd7c08d-5d09-486b-845b-c1fad568ba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:39.113898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:39.160055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 433/890] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:39.183862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4595a-eb6c-4844-92bb-1d3a50c952b2 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:39.291540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4595a-eb6c-4844-92bb-1d3a50c952b2 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:39.310049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:39.327075 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:39.327908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.328182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.328465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.329156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:39.330518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:39.342577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:39.343523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:39.344415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 458/891] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Jun 16 00:17:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:39.350222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7c08d-5d09-486b-845b-c1fad568ba30 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:39.453865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7c08d-5d09-486b-845b-c1fad568ba30 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:39.489392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc8b21-eb73-402e-81cd-b691673e2ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:39.490409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8b21-eb73-402e-81cd-b691673e2ea9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.490899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8b21-eb73-402e-81cd-b691673e2ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.491422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8b21-eb73-402e-81cd-b691673e2ea9 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.492922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc8b21-eb73-402e-81cd-b691673e2ea9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:39.493449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc8b21-eb73-402e-81cd-b691673e2ea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:39.495023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc8b21-eb73-402e-81cd-b691673e2ea9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:39.501761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 434/892] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:39.507195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:39.624216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915032-7d17-448a-a60f-de5716a9ae84 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:39.657062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea5768-b81d-4485-9f34-75415685ebc9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.657498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea5768-b81d-4485-9f34-75415685ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.657782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea5768-b81d-4485-9f34-75415685ebc9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.658226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:39.659074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.659523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.659961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.660727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 435/893] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:39.661225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:39.661692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:39.663285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:39.668196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308e7f-5e31-4488-a11a-09723f1a71af None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.668484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308e7f-5e31-4488-a11a-09723f1a71af None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.668839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308e7f-5e31-4488-a11a-09723f1a71af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.670377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 459/894] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:39.678475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:39.690985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-5d308e7f-5e31-4488-a11a-09723f1a71af None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:39.692183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5d308e7f-5e31-4488-a11a-09723f1a71af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:39.717407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:39.728748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:39.772154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 436/895] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:39.789870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d308e7f-5e31-4488-a11a-09723f1a71af None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:39.801199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168be05-39d5-498f-aa96-7beb7f4223fd None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:39.839438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:39.840357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.840803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.841251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.842149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] RBAC: Adding query filter params (interface=public, service_id=9298dca1b221480dabac9cd3ab890812, region_id=RegionOne) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:39.842572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:39.844155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:39.855126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 460/896] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jun 16 00:17:39 2022] GET /identity/v3/endpoints?service_id=9298dca1b221480dabac9cd3ab890812&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:39.856230 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:39.891008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d308e7f-5e31-4488-a11a-09723f1a71af None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:39.931490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c575-47e5-42c3-bc60-f064d444d9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:39.932432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c575-47e5-42c3-bc60-f064d444d9f9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:39.932757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c575-47e5-42c3-bc60-f064d444d9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:39.933385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c575-47e5-42c3-bc60-f064d444d9f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:39.944378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4c575-47e5-42c3-bc60-f064d444d9f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-7db4c575-47e5-42c3-bc60-f064d444d9f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:39.953143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:39.954183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:17:39.955145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:39.959321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 437/897] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jun 16 00:17:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:39.961000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c575-47e5-42c3-bc60-f064d444d9f9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:39.969622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329826bc-6bec-46de-9526-d2be8c9bb6e9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:39.972205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:39.993039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:40.019200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75991-74cc-4578-aa3e-ad9e3d1c67c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:40.020793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75991-74cc-4578-aa3e-ad9e3d1c67c3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.021276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75991-74cc-4578-aa3e-ad9e3d1c67c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.022026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75991-74cc-4578-aa3e-ad9e3d1c67c3 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.023720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b75991-74cc-4578-aa3e-ad9e3d1c67c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:40.025564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b75991-74cc-4578-aa3e-ad9e3d1c67c3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:40.036619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 461/898] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:39 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:17:40.091741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c575-47e5-42c3-bc60-f064d444d9f9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:40.131789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:40.133428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.133927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.134589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.138181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:40.139965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:40.141678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:40.152826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 438/899] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jun 16 00:17:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:40.161696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75991-74cc-4578-aa3e-ad9e3d1c67c3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:40.279722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75991-74cc-4578-aa3e-ad9e3d1c67c3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:40.317147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:40.318394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.319024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.319537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.330926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:40.332672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:40.352357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:40.353461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:40.354416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:40.354416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:40.354416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:40.354416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:40.356763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 462/900] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:40.362696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:40.445707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:40.463897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aac42-2251-46f8-b6e2-e68f1488cd9c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:40.498201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba99f3-6536-4912-a431-445fd3ec7ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:40.498863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba99f3-6536-4912-a431-445fd3ec7ceb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.499149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba99f3-6536-4912-a431-445fd3ec7ceb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.499429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba99f3-6536-4912-a431-445fd3ec7ceb None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.500147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba99f3-6536-4912-a431-445fd3ec7ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:40.500453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba99f3-6536-4912-a431-445fd3ec7ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:40.501828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba99f3-6536-4912-a431-445fd3ec7ceb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:40.514183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 439/901] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:40.519617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:40.541555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:40.631199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544a5cf-b4e5-43cf-87fc-efdcdbc50b22 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:40.650506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd07a-1fe3-4676-9717-52f1e425df27 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.650808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd07a-1fe3-4676-9717-52f1e425df27 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.651074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd07a-1fe3-4676-9717-52f1e425df27 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.652596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 440/902] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:40.657377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db206-3396-460d-9108-344d599370c5 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.657672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db206-3396-460d-9108-344d599370c5 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.657959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db206-3396-460d-9108-344d599370c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.666724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:40.667545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.667828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.668112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.675838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:40.676646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-501db206-3396-460d-9108-344d599370c5 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:40.677304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:40.677839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-501db206-3396-460d-9108-344d599370c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:40.693829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:40.695433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:40.696466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 463/903] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:40.701262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:40.768790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 441/904] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:40.788832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db206-3396-460d-9108-344d599370c5 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:40.792907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:40.813104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c446d89-f353-4097-b181-95536c415d87 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:40.851712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:40.852507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.852984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.853339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.854355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:40.854737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:40.856255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:40.860931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:40.882038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 464/905] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jun 16 00:17:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:40.883117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:40.895055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db206-3396-460d-9108-344d599370c5 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:40.931950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d79e6-0bfe-4bb5-8b77-701c43cfbd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:40.932830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d79e6-0bfe-4bb5-8b77-701c43cfbd9c None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:40.933232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d79e6-0bfe-4bb5-8b77-701c43cfbd9c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:40.933576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d79e6-0bfe-4bb5-8b77-701c43cfbd9c None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:40.934419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d79e6-0bfe-4bb5-8b77-701c43cfbd9c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:40.935855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d79e6-0bfe-4bb5-8b77-701c43cfbd9c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:40.986721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e3e79-32df-4a49-9d62-cb0c6622bc55 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.008695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 442/906] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:40 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:17:41.040130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe896c-a8c6-4bc5-82c8-ed96f0e61538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.040869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe896c-a8c6-4bc5-82c8-ed96f0e61538 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.041166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe896c-a8c6-4bc5-82c8-ed96f0e61538 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.041445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe896c-a8c6-4bc5-82c8-ed96f0e61538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.044932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.118734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe896c-a8c6-4bc5-82c8-ed96f0e61538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.120437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe896c-a8c6-4bc5-82c8-ed96f0e61538 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.237316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 465/907] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:40 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:41.242848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d79e6-0bfe-4bb5-8b77-701c43cfbd9c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.295276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d7c0-1d9a-45ba-a744-c10ab82a6d75 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.295944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d7c0-1d9a-45ba-a744-c10ab82a6d75 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.296497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d7c0-1d9a-45ba-a744-c10ab82a6d75 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.298341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 466/908] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:41.307021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ca441-6d04-437b-af95-1da72376a77f None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.307310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ca441-6d04-437b-af95-1da72376a77f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.307588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ca441-6d04-437b-af95-1da72376a77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.329624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-789ca441-6d04-437b-af95-1da72376a77f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:41.330814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-789ca441-6d04-437b-af95-1da72376a77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:41.359334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d79e6-0bfe-4bb5-8b77-701c43cfbd9c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.371555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.389705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.395909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.396861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.397276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.397654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.409188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.410833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.424828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 467/909] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:41.436664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:41.437582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:41.438494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:41.438494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:41.438494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:41.438494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:41.438494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 443/910] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:41.443631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ca441-6d04-437b-af95-1da72376a77f None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.449510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.557444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ca441-6d04-437b-af95-1da72376a77f None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.582820 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e3bca-92c5-49d5-a482-83d6fa1ecfff None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.607300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.607994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.608321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.608637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.609374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:41.609686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.611080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.632997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 468/911] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:41.634740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.636633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.637435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.637712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.637982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.638633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.640228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:41.656718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:41.657908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:17:41.658802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:41.663065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 444/912] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:41.663942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.665214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.716949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.744685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c38439-3e2e-4621-9362-e8acd77c8bd5 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.772758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81ca20-d895-4fff-b905-ccf34cfa33d0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.781163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.782082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.782470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.782850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.790063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.791750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:41.807740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:41.809332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:41.810466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 469/913] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:41.812486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.813196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.813480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.813758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.814491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:41.814800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.816218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.816913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.828190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 445/914] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:41.829014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.915026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.935813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f38c0a-8fc4-4d0a-b88b-e4de628cae99 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.941740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694057eb-47de-4a6a-a274-d8c656be8524 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.952900 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.976370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.977307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.977953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.978378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.979229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:41.979680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.980039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.980794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.981137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.981479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.982734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.982854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:41.982854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.984670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.994843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 446/915] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:41.996859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.000042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 470/916] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:42.005597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.117823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3c74f-457e-4e9d-825a-c214449791e8 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.132942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2097fcb-ec90-4b88-aaf9-a7f1cc387a93 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.166636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067aaa8e-4efa-4ad6-9cb5-decb33c1e7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.167692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aaa8e-4efa-4ad6-9cb5-decb33c1e7de None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.168177 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aaa8e-4efa-4ad6-9cb5-decb33c1e7de None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.168698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aaa8e-4efa-4ad6-9cb5-decb33c1e7de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.169743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aaa8e-4efa-4ad6-9cb5-decb33c1e7de None admin] RBAC: Adding query filter params (role.id=aaf93b32d93946789fbde39f351cbc44, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=396cdfa4dbf54e66919bc3e054c54154) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:42.170262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aaa8e-4efa-4ad6-9cb5-decb33c1e7de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.170383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aef482-d914-4433-afb1-1dae6d299260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.170986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aef482-d914-4433-afb1-1dae6d299260 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.171276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aef482-d914-4433-afb1-1dae6d299260 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.171557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aef482-d914-4433-afb1-1dae6d299260 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.172303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aef482-d914-4433-afb1-1dae6d299260 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.172866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aaa8e-4efa-4ad6-9cb5-decb33c1e7de None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.173930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aef482-d914-4433-afb1-1dae6d299260 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.180906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 471/917] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/role_assignments?user.id=396cdfa4dbf54e66919bc3e054c54154&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=aaf93b32d93946789fbde39f351cbc44 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:42.267105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 447/918] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:17:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:17:42.330036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f1572-4cc3-4d3b-8a96-adb5cd284820 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.330408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f1572-4cc3-4d3b-8a96-adb5cd284820 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.330592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f1572-4cc3-4d3b-8a96-adb5cd284820 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.332189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 472/919] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:42.339028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68cd8-939c-4a95-880e-adb704f20793 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.339349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68cd8-939c-4a95-880e-adb704f20793 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.339583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68cd8-939c-4a95-880e-adb704f20793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.359048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-33f68cd8-939c-4a95-880e-adb704f20793 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:42.360273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-33f68cd8-939c-4a95-880e-adb704f20793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:42.389368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:42.448658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 448/920] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:42.466380 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f1572-4cc3-4d3b-8a96-adb5cd284820 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.480841 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:42.580139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f1572-4cc3-4d3b-8a96-adb5cd284820 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.622846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.623970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.624440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.625021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.625861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.627411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:42.640188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:42.641307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:17:42.642203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:42.648639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 473/921] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:42.650986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f68cd8-939c-4a95-880e-adb704f20793 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.755276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f68cd8-939c-4a95-880e-adb704f20793 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.790638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.791553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.791993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.792422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.793365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:42.793851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.795423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.804643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 449/922] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:42.807210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.888303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:42.921164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722a03e-3220-40c6-961e-0c103bb1dbdd None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.956093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41287a2e-34b3-419a-b174-d056bed27acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.957079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41287a2e-34b3-419a-b174-d056bed27acb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.957512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41287a2e-34b3-419a-b174-d056bed27acb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.957934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41287a2e-34b3-419a-b174-d056bed27acb None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.958890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41287a2e-34b3-419a-b174-d056bed27acb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:42.959356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41287a2e-34b3-419a-b174-d056bed27acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.961408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41287a2e-34b3-419a-b174-d056bed27acb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.971403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 474/923] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:42.972981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.999505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:43.082056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f398-506b-467d-bf3a-59a64f0a1c69 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:43.117737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73a051-b26c-451a-a438-353fd2fc17a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:43.118401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a051-b26c-451a-a438-353fd2fc17a5 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:43.118685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a051-b26c-451a-a438-353fd2fc17a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:43.118962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a051-b26c-451a-a438-353fd2fc17a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:43.119718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73a051-b26c-451a-a438-353fd2fc17a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:43.121115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73a051-b26c-451a-a438-353fd2fc17a5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:43.214637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 450/924] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:17:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:43.404315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39796ae-32bd-4ed8-86a3-9032cb112d50 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:43.404832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39796ae-32bd-4ed8-86a3-9032cb112d50 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:43.405222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39796ae-32bd-4ed8-86a3-9032cb112d50 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:43.406937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 475/925] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:43.412305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8789f4-722a-42c9-811e-1c8b28b77f16 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:43.412627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8789f4-722a-42c9-811e-1c8b28b77f16 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:43.412904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8789f4-722a-42c9-811e-1c8b28b77f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:43.432696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-fc8789f4-722a-42c9-811e-1c8b28b77f16 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:43.433882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fc8789f4-722a-42c9-811e-1c8b28b77f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:43.511578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 451/926] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:43.534039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39796ae-32bd-4ed8-86a3-9032cb112d50 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:43.640806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39796ae-32bd-4ed8-86a3-9032cb112d50 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:43.675156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:43.678720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:43.678720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:43.678720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:43.678720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:43.678720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:43.681760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:43.692053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:43.693105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:43.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 476/927] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:43.700036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8789f4-722a-42c9-811e-1c8b28b77f16 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:43.702458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:43.801303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8789f4-722a-42c9-811e-1c8b28b77f16 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:43.838064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:43.838732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:43.839014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:43.839293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:43.839997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:43.840303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:43.841700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:43.846787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 452/928] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:43.853066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:43.956832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa07ed-4049-488e-ba21-4f7de0237231 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:43.992915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:43.994113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:43.997049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:43.997639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:43.999686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:44.000211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:44.001839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:44.008080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 477/929] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:44.013661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:44.086075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:44.122744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e00e4-efb5-452c-93cf-a8aabb0742c6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:44.157619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7c09-8fc7-4940-9385-56c3007d4992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:44.158632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7c09-8fc7-4940-9385-56c3007d4992 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.159154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7c09-8fc7-4940-9385-56c3007d4992 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.159635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7c09-8fc7-4940-9385-56c3007d4992 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.160731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc7c09-8fc7-4940-9385-56c3007d4992 None admin] RBAC: Adding query filter params (interface=public, service_id=4a0a1c718d60456b90407bd1f6048e36, region_id=RegionOne) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:44.161260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc7c09-8fc7-4940-9385-56c3007d4992 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:44.163005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc7c09-8fc7-4940-9385-56c3007d4992 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:44.173483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 453/930] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jun 16 00:17:44 2022] GET /identity/v3/endpoints?service_id=4a0a1c718d60456b90407bd1f6048e36&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:44.181670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:44.245462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:44.286226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f365d68-9dfe-4ee7-97a5-936f049d8949 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:44.322628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a728047-1223-44b3-8422-3d243d9bb90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:44.323715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a728047-1223-44b3-8422-3d243d9bb90c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.324206 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a728047-1223-44b3-8422-3d243d9bb90c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.324717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a728047-1223-44b3-8422-3d243d9bb90c None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.325640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a728047-1223-44b3-8422-3d243d9bb90c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:44.327271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a728047-1223-44b3-8422-3d243d9bb90c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:44.335051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 478/931] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:44 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:17:44.409761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1762498-ac44-426a-a43c-ebc9392fc80c None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.410380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1762498-ac44-426a-a43c-ebc9392fc80c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.410867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1762498-ac44-426a-a43c-ebc9392fc80c None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.412886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 454/932] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:44.418358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68c57-6d69-4c9c-9401-9fa6d0995819 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.419009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68c57-6d69-4c9c-9401-9fa6d0995819 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.419508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68c57-6d69-4c9c-9401-9fa6d0995819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.439271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a1b68c57-6d69-4c9c-9401-9fa6d0995819 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:44.440868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a1b68c57-6d69-4c9c-9401-9fa6d0995819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:44.519110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 479/933] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:44.542431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1762498-ac44-426a-a43c-ebc9392fc80c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:44.605033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ba4bb-a885-4ba9-bcf5-352f56a5cea5 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.605649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ba4bb-a885-4ba9-bcf5-352f56a5cea5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.606133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ba4bb-a885-4ba9-bcf5-352f56a5cea5 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.607970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 480/934] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:44.613054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16950f35-ae39-4806-8694-d72a454938df None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.613583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16950f35-ae39-4806-8694-d72a454938df None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.614067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16950f35-ae39-4806-8694-d72a454938df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.631131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-16950f35-ae39-4806-8694-d72a454938df None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:44.632740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-16950f35-ae39-4806-8694-d72a454938df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:44.661306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1762498-ac44-426a-a43c-ebc9392fc80c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:44.690288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:44.700346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac836147-e272-4126-ace4-da3512a283cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:44.701425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836147-e272-4126-ace4-da3512a283cc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.701916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836147-e272-4126-ace4-da3512a283cc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.702381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836147-e272-4126-ace4-da3512a283cc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.712602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 481/935] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:44.712873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:44.718010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac836147-e272-4126-ace4-da3512a283cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:44.730021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16950f35-ae39-4806-8694-d72a454938df None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-ac836147-e272-4126-ace4-da3512a283cc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:44.731070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:44.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:17:44.733293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:44.733949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 455/936] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jun 16 00:17:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:44.739145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac836147-e272-4126-ace4-da3512a283cc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:44.834559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16950f35-ae39-4806-8694-d72a454938df None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:44.847949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac836147-e272-4126-ace4-da3512a283cc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:44.872738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:44.873729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.874227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.874719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.875714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:44.877375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:44.878965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING keystone.server.flask.common [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 00:17:44.892739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:44.893419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:44.893968 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:44.894261 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:44.895059 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:44.895936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:44.897477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:44.906691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 456/937] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jun 16 00:17:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:44.912198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:44.921971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.sql.core [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 00:17:44.921971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 00:17:44.921971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [parameters: {'id': '1a408651d3784668ae97a22457a36476', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 00:17:44.921971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86686) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a408651d3784668ae97a22457a36476.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a408651d3784668ae97a22457a36476. Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:17:44.942623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result.read() Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 00:17:44.943562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application yield resource Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise self.value Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application session.commit() Jun 16 00:17:44.944649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.session.flush() Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application compat.raise_( Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 00:17:44.946105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 00:17:44.947029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result.read() Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:17:44.947854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [parameters: {'id': '1a408651d3784668ae97a22457a36476', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:44.950208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:44.952215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a408651d3784668ae97a22457a36476. Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 482/938] 10.222.0.20 () {60 vars in 1140 bytes} [Thu Jun 16 00:17:44 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:44.954460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:44.992632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.013187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.055673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4749-9ac6-451b-9a20-c52f0b3dbae8 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.082905 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db86f-92e2-40ae-a272-f3677f2e8323 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.102743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.103844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.104285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.105267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.120449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.121088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.121531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.121917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.122295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.123391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.129689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.131782 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:45.144271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:45.145428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:45.146615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:45.146615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:45.146615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:45.146615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:45.148533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 457/939] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:45.153737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:45.154826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:45.155942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:45.155942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:45.155942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:45.155942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:45.155942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:45.155942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:45.155942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:45.155942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:45.156600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.163101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 483/940] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:45.163906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.278772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696e1c2-b0cd-4487-9e49-ec274c1f48eb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.282773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.285114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f1394-4397-4f57-9ec8-f5522945a41f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.288258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.323220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.324294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.324879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.325421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.326541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:45.327010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.327133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c601e51-3c66-4a33-9407-4dc0a2be27d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.327940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601e51-3c66-4a33-9407-4dc0a2be27d2 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.328337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601e51-3c66-4a33-9407-4dc0a2be27d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.328786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601e51-3c66-4a33-9407-4dc0a2be27d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.329202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.329819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c601e51-3c66-4a33-9407-4dc0a2be27d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:45.330273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c601e51-3c66-4a33-9407-4dc0a2be27d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.331793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c601e51-3c66-4a33-9407-4dc0a2be27d2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.343998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 458/941] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:45.349180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.360416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 484/942] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:45.461711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026e1e4-8757-4614-8867-70941c82adef None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.503826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.504699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.504991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.505271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.514228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.516051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:45.532245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:45.533356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:45.534325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:45.537176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 459/943] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:45.544553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902420fc-af56-4636-b278-1f3df7076736 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.544553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902420fc-af56-4636-b278-1f3df7076736 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.544553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902420fc-af56-4636-b278-1f3df7076736 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.544553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 485/944] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:45.544952 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.549780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091177b-db04-41d4-9798-71f8fb3bde7e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.550080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091177b-db04-41d4-9798-71f8fb3bde7e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.550368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091177b-db04-41d4-9798-71f8fb3bde7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.573387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a091177b-db04-41d4-9798-71f8fb3bde7e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:45.575475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a091177b-db04-41d4-9798-71f8fb3bde7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:45.579223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.671496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404559a-13eb-4c4a-a876-36b7620657fc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.682495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 486/945] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:45.705077 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091177b-db04-41d4-9798-71f8fb3bde7e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.714899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.714899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.714899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.714899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.715276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:45.715276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.716464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.737160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 460/946] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:45.742538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.826800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.839571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091177b-db04-41d4-9798-71f8fb3bde7e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.850660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.870945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8d90-2cae-4919-9dbe-7283e5e8a2b4 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.884096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.885253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.885722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.886180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] PATH_INFO: `/v3/services/compute` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.887043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.888619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:45.902749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:45.903837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:45.904755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:45.907506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 487/947] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:45.912402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.913281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.914149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.914425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.916219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.929575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.931637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:45.950753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:45.951908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:45.953121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:45.953121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:45.953121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:45.953121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:45.955194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 461/948] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:45.963359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.049411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626886fb-d6c5-4f12-bac8-c702bbca725b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.084454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7601c9f-3cc5-450c-a988-2923a5cf17bc None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.084994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.086347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.086728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.087140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.087997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:46.088402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.089881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.095443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 488/949] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:46.100633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.122197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.122869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.123160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.123443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.124175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:46.124492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.125877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.127303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.133271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.139612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 462/950] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:46.152544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.234771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4a23-1a87-4c50-927e-cbe059802ef9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.261761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb72ffa-ea72-47a6-aa6e-2a99e6f48124 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.279578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.280417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.280753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.281328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.282065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:46.282382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.283817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.290677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 489/951] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:46.310157 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.310898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.311187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.311468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.320316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.321850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:46.338588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:46.339623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:46.340624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 463/952] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:46.346181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.443084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.465355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d0bb-788e-43b2-bb77-25e224651690 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.506148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ab44c-e89c-4a47-b198-19e5c3e1c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.506148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab44c-e89c-4a47-b198-19e5c3e1c25f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.506148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab44c-e89c-4a47-b198-19e5c3e1c25f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.506148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab44c-e89c-4a47-b198-19e5c3e1c25f None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.506588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ab44c-e89c-4a47-b198-19e5c3e1c25f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:46.506862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ab44c-e89c-4a47-b198-19e5c3e1c25f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.509180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ab44c-e89c-4a47-b198-19e5c3e1c25f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.522851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 490/953] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:46.529276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.568433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95247146-9146-4c98-81ed-29f5b547f225 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.568750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95247146-9146-4c98-81ed-29f5b547f225 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.568939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95247146-9146-4c98-81ed-29f5b547f225 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.571207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 491/954] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:46.579489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db48fd2-ef3a-4872-8baf-5ac29ae7819d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.579489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db48fd2-ef3a-4872-8baf-5ac29ae7819d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.579489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db48fd2-ef3a-4872-8baf-5ac29ae7819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.600113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6db48fd2-ef3a-4872-8baf-5ac29ae7819d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:46.601595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6db48fd2-ef3a-4872-8baf-5ac29ae7819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:46.606905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.658727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a127-8663-4bdc-abef-2cbc021b9eb9 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.688213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 492/955] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:46.702350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e016b-8575-419b-b9f0-d91db3a90ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.703510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e016b-8575-419b-b9f0-d91db3a90ec3 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.703994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e016b-8575-419b-b9f0-d91db3a90ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.704497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e016b-8575-419b-b9f0-d91db3a90ec3 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/396cdfa4dbf54e66919bc3e054c54154/roles/aaf93b32d93946789fbde39f351cbc44` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.706998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e016b-8575-419b-b9f0-d91db3a90ec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=396cdfa4dbf54e66919bc3e054c54154, role_id=aaf93b32d93946789fbde39f351cbc44)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.708764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e016b-8575-419b-b9f0-d91db3a90ec3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.720373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db48fd2-ef3a-4872-8baf-5ac29ae7819d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.792516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.808661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 464/956] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:46 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/396cdfa4dbf54e66919bc3e054c54154/roles/aaf93b32d93946789fbde39f351cbc44 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:46.823202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db48fd2-ef3a-4872-8baf-5ac29ae7819d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.861494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.862306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.862590 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.862858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.863514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.864972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:46.878795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:46.879985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:46.880944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 493/957] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:46.886401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e016b-8575-419b-b9f0-d91db3a90ec3 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:47.000582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e016b-8575-419b-b9f0-d91db3a90ec3 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:47.018732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:47.039794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:47.040460 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:47.040773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:47.041052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:47.041769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:47.042076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:47.043433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:47.049481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 465/958] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:47.055637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:47.158689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca3b4c-e6c8-4cd9-ac35-17f15101cb30 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:47.193918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdf859-666b-4e30-969a-7d0255947a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:47.194584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdf859-666b-4e30-969a-7d0255947a5a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:47.194943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdf859-666b-4e30-969a-7d0255947a5a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:47.195222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdf859-666b-4e30-969a-7d0255947a5a None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:47.195929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdf859-666b-4e30-969a-7d0255947a5a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:47.196235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdf859-666b-4e30-969a-7d0255947a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:47.197681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdf859-666b-4e30-969a-7d0255947a5a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:47.202931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 494/959] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:47.208576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:47.312895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0af0c-b2c2-4ff0-b9ae-4af8fd7a0ec8 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:47.348289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03449a8c-4a2a-4afa-a069-30939b3cf7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:47.349364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03449a8c-4a2a-4afa-a069-30939b3cf7db None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:47.349854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03449a8c-4a2a-4afa-a069-30939b3cf7db None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:47.350357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03449a8c-4a2a-4afa-a069-30939b3cf7db None admin] PATH_INFO: `/v3/endpoints` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:47.351888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03449a8c-4a2a-4afa-a069-30939b3cf7db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:47.353495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03449a8c-4a2a-4afa-a069-30939b3cf7db None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:47.452756 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 466/960] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:17:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:47.804686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82432e27-ba00-47cc-842b-423bf165089d None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:47.805070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82432e27-ba00-47cc-842b-423bf165089d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:47.805276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82432e27-ba00-47cc-842b-423bf165089d None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:47.806870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 495/961] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:47.812048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80fbef-74f1-484d-ad62-9230aa5024d0 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:47.812392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80fbef-74f1-484d-ad62-9230aa5024d0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:47.812705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80fbef-74f1-484d-ad62-9230aa5024d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:47.829798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9a80fbef-74f1-484d-ad62-9230aa5024d0 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:47.831249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9a80fbef-74f1-484d-ad62-9230aa5024d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:47.860191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:47.909599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 467/962] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:47.936237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82432e27-ba00-47cc-842b-423bf165089d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:47.940223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:48.062173 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82432e27-ba00-47cc-842b-423bf165089d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:48.097734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:48.098412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:48.098696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:48.099092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:48.104071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:48.105904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:48.156244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.sql.core [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 00:17:48.156244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 00:17:48.156244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [parameters: {'user_id': 'b76d2812e75f4687b841f4b4b1f2c1d0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 00:17:48.156244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86686) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result.read() Jun 16 00:17:48.164446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:17:48.165346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application yield resource Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise self.value Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application session.commit() Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 00:17:48.166324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.session.flush() Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application compat.raise_( Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 00:17:48.167251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:17:48.168163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application result.read() Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:17:48.169090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b76d2812e75f4687b841f4b4b1f2c1d0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:48.169944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 00:17:48.170761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 00:17:48.171705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 00:17:48.171705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 00:17:48.171705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 00:17:48.171705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 00:17:48.171705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 00:17:48.171705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:48.171705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 496/963] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jun 16 00:17:47 2022] POST /identity/v3/users => generated 160 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:48.173426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80fbef-74f1-484d-ad62-9230aa5024d0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:48.283455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80fbef-74f1-484d-ad62-9230aa5024d0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:48.320240 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c426d7-47e4-457b-8c83-a226e9e1cf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:48.320999 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c426d7-47e4-457b-8c83-a226e9e1cf2d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:48.321288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c426d7-47e4-457b-8c83-a226e9e1cf2d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:48.321570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c426d7-47e4-457b-8c83-a226e9e1cf2d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:48.329050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c426d7-47e4-457b-8c83-a226e9e1cf2d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:48.330488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c426d7-47e4-457b-8c83-a226e9e1cf2d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d8c426d7-47e4-457b-8c83-a226e9e1cf2d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:48.345964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:48.348169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:48.350452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 468/964] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:48.356565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:48.424945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:48.468484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a816326-f09b-4f6f-a24a-708313465878 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:48.495957 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5a91-1913-4dee-b57c-318a5d1c2ac6 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:48.498141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5a91-1913-4dee-b57c-318a5d1c2ac6 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:48.498141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5a91-1913-4dee-b57c-318a5d1c2ac6 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:48.498141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 469/965] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:48.503237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51b73c-0bf9-4a4c-b75a-e0a8a3bb5200 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:48.503535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51b73c-0bf9-4a4c-b75a-e0a8a3bb5200 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:48.503821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51b73c-0bf9-4a4c-b75a-e0a8a3bb5200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:48.506755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:48.507460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:48.507743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:48.508020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:48.508868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:48.509280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:48.510923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:48.521341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ca51b73c-0bf9-4a4c-b75a-e0a8a3bb5200 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:48.522876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ca51b73c-0bf9-4a4c-b75a-e0a8a3bb5200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:48.540560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:48.567959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 497/966] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jun 16 00:17:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:48.604680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 470/967] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:48.629138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:48.745098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6e9b4-4d7c-4796-997f-15745ce704c9 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:48.771494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:48.782806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0565f-6ac5-4c73-834a-6990e4b5b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:48.783481 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0565f-6ac5-4c73-834a-6990e4b5b4af None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:48.783768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0565f-6ac5-4c73-834a-6990e4b5b4af None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:48.784108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0565f-6ac5-4c73-834a-6990e4b5b4af None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:48.784878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0565f-6ac5-4c73-834a-6990e4b5b4af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:48.786256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0565f-6ac5-4c73-834a-6990e4b5b4af None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:48.866263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 498/968] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:48 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:49.075972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64e1bf-fef9-42a3-b240-1f385d845ff4 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.076330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64e1bf-fef9-42a3-b240-1f385d845ff4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.076645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64e1bf-fef9-42a3-b240-1f385d845ff4 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.078169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 471/969] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:49.083397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d128-b45f-4417-ba39-efdca32bcabd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.083837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d128-b45f-4417-ba39-efdca32bcabd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.084019 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d128-b45f-4417-ba39-efdca32bcabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.101471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7897d128-b45f-4417-ba39-efdca32bcabd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:49.102968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7897d128-b45f-4417-ba39-efdca32bcabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:49.177056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 499/970] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:49.200656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64e1bf-fef9-42a3-b240-1f385d845ff4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.302581 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64e1bf-fef9-42a3-b240-1f385d845ff4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:49.337635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30dff7-d521-47c4-8734-f8cb27d86355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:49.338449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30dff7-d521-47c4-8734-f8cb27d86355 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.338733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30dff7-d521-47c4-8734-f8cb27d86355 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.339010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30dff7-d521-47c4-8734-f8cb27d86355 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.349328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30dff7-d521-47c4-8734-f8cb27d86355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-ce30dff7-d521-47c4-8734-f8cb27d86355 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:49.357478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:49.358500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:17:49.359439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:49.363518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 472/971] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:49.364818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897d128-b45f-4417-ba39-efdca32bcabd None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.466763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897d128-b45f-4417-ba39-efdca32bcabd None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:49.477769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:49.501085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:49.501830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.502111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.502391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.503143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:49.503456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:49.504839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:49.518662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 500/972] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:49.520260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30dff7-d521-47c4-8734-f8cb27d86355 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.533526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:49.626031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30dff7-d521-47c4-8734-f8cb27d86355 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:49.663474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176400-d927-401b-ac80-b1caeaf3e779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:49.664294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176400-d927-401b-ac80-b1caeaf3e779 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.664608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176400-d927-401b-ac80-b1caeaf3e779 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.664892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176400-d927-401b-ac80-b1caeaf3e779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.677407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176400-d927-401b-ac80-b1caeaf3e779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:49.678865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176400-d927-401b-ac80-b1caeaf3e779 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d2176400-d927-401b-ac80-b1caeaf3e779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:49.698050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:49.704703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:49.705636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:49.705636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:49.705636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:49.705636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:49.705636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 473/973] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:49.705948 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.713006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ffa3-3a65-4f1e-bc9c-aadd53d9d7b6 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.713302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ffa3-3a65-4f1e-bc9c-aadd53d9d7b6 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.713589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ffa3-3a65-4f1e-bc9c-aadd53d9d7b6 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.714945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 474/974] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:49.720103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db04db-5090-4bb6-a1b7-baf078b2a5f1 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.720340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db04db-5090-4bb6-a1b7-baf078b2a5f1 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.720732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db04db-5090-4bb6-a1b7-baf078b2a5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.741082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-50db04db-5090-4bb6-a1b7-baf078b2a5f1 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:49.742804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-50db04db-5090-4bb6-a1b7-baf078b2a5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:49.799926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:49.811243 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 475/975] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:49.816637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d413689-a901-469e-8408-7c87d8ad4338 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:49.834099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db04db-5090-4bb6-a1b7-baf078b2a5f1 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.859916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:49.861522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.861522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.861522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.863086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:49.863404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:49.867780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:49.885890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 501/976] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:49.887690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.922930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:49.945492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db04db-5090-4bb6-a1b7-baf078b2a5f1 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:49.982811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:49.983659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.984013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.984301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.985000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:49.986442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:49.998098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46f7f4-4f27-4335-9c5a-ded63a4a336d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:50.000144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:50.001370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:50.005675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 476/977] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:50.008577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:50.040594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.041409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.041686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.041959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.053965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.056208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:50.072062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:50.073207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:17:50.074220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:50.074779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:50.074855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 502/978] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:50.078886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:50.122421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4c1fc-ebbf-47d2-9420-e990480242c0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.164156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.165335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.165636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.165920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.166635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:50.166947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.168432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.171517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:50.179498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 477/979] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:50.180497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:50.181430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cf553-e7bd-4a9b-8433-e51f3f7dd01d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.216686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.217660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.218140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.218627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.220178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:50.220780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.223440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.244579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 503/980] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:50.249994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:50.294111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29bde-0279-4986-a192-107dcaeb4bd6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.314679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:50.329470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.330144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.330410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.330690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.331387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:50.331692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.333076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.338921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 478/981] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:50.344830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:50.357866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232feab-6f56-4bf7-888a-c62b55ded697 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.393142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ec0bc-6f78-4343-9e16-fd35b192999f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.393797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ec0bc-6f78-4343-9e16-fd35b192999f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.394082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ec0bc-6f78-4343-9e16-fd35b192999f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.394360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ec0bc-6f78-4343-9e16-fd35b192999f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.415624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:50.446100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837b2c4-74da-46df-ab20-1381aa10dc49 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.469457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ec0bc-6f78-4343-9e16-fd35b192999f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.470983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ec0bc-6f78-4343-9e16-fd35b192999f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.481731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.482591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.482938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.483285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.484153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] RBAC: Adding query filter params (interface=admin, service_id=4a0a1c718d60456b90407bd1f6048e36, region_id=RegionOne) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:50.484549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.485905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.490986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 479/982] 10.222.0.20 () {56 vars in 1249 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/endpoints?service_id=4a0a1c718d60456b90407bd1f6048e36&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:50.496472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:50.546937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:50.581831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 504/983] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:17:50.587341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ec0bc-6f78-4343-9e16-fd35b192999f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:50.606140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7c3c7-9736-41c4-ad7e-68aa70359dc2 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.639867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:50.640450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1855160-b0e0-4c3e-a865-5a2f2d34afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.641197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1855160-b0e0-4c3e-a865-5a2f2d34afd1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.641477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1855160-b0e0-4c3e-a865-5a2f2d34afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.641753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1855160-b0e0-4c3e-a865-5a2f2d34afd1 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.642434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1855160-b0e0-4c3e-a865-5a2f2d34afd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.643782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1855160-b0e0-4c3e-a865-5a2f2d34afd1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.650420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 480/984] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:17:50.697316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ec0bc-6f78-4343-9e16-fd35b192999f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.734663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.735474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.735853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.736132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.748955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.750413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:50.768663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:50.769658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:50.770584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:50.770584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:50.770584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:50.770584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:50.770584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 505/985] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:50.776672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1855160-b0e0-4c3e-a865-5a2f2d34afd1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:50.880790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1855160-b0e0-4c3e-a865-5a2f2d34afd1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:50.893506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:50.914905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68984f8-6a91-4b6f-9c2d-1d5b415fcf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:50.915743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68984f8-6a91-4b6f-9c2d-1d5b415fcf60 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:50.916031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68984f8-6a91-4b6f-9c2d-1d5b415fcf60 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:50.916363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68984f8-6a91-4b6f-9c2d-1d5b415fcf60 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:50.917278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68984f8-6a91-4b6f-9c2d-1d5b415fcf60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:50.917583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68984f8-6a91-4b6f-9c2d-1d5b415fcf60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:50.919450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68984f8-6a91-4b6f-9c2d-1d5b415fcf60 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:50.932243 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 481/986] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:50.937897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:51.033725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06024a-9e64-4771-914f-01b8ed12f3f9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.034086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06024a-9e64-4771-914f-01b8ed12f3f9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.034372 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06024a-9e64-4771-914f-01b8ed12f3f9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.035984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 482/987] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:51.041796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcc7dd-52ab-4d8b-bfef-8aba41dae034 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.042083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcc7dd-52ab-4d8b-bfef-8aba41dae034 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.042359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcc7dd-52ab-4d8b-bfef-8aba41dae034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.046488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1722-ddf9-4711-89a3-b75f8335bdaa None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:51.048180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:51.059758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-5fbcc7dd-52ab-4d8b-bfef-8aba41dae034 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:51.061015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5fbcc7dd-52ab-4d8b-bfef-8aba41dae034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:51.083233 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:51.084030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.084327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.084631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.092302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:51.093774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:51.107958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:51.109134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:51.110068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 506/988] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:51.114703 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:51.135439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 483/989] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:51.158450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcc7dd-52ab-4d8b-bfef-8aba41dae034 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:51.225655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290b1f6-cf7d-4652-babd-11efd5b20cdd None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:51.261243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:51.261708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcc7dd-52ab-4d8b-bfef-8aba41dae034 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:51.261913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.262203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.262484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.263277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:51.263594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:51.263977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:51.265426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:51.278052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 507/990] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:51.283840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:51.299630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:51.300433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.300752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.301033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.301697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:51.303060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:51.309419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:51.315859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:51.318230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:51.319108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 484/991] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:51.324026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:51.393566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e6913-3db2-4f0d-9496-c69f536ce209 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:51.428096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:51.428786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.429103 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.429382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.430224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] RBAC: Adding query filter params (role.id=aaf93b32d93946789fbde39f351cbc44, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=396cdfa4dbf54e66919bc3e054c54154) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:51.430537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:51.431962 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:51.437822 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 508/992] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:51 2022] GET /identity/v3/role_assignments?user.id=396cdfa4dbf54e66919bc3e054c54154&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=aaf93b32d93946789fbde39f351cbc44 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:51.438953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fb692-4794-4271-8d75-5e5887c729a7 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:51.486517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:51.487214 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.487499 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.487778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.488488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] RBAC: Adding query filter params (name=compute) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:51.488828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:51.490241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:51.496646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 485/993] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:51.502574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:51.602603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:51.618125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077803-c2cc-44e4-a6bb-0e66621fd123 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:51.653901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:51.654567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.654854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.655132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.655838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] RBAC: Adding query filter params (type=compute) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:51.656142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:51.657538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:51.662976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 509/994] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:51.669355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:51.702112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:51.774686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e367b-2a34-4324-984e-376be187b1eb None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:51.809715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda131f5-9a42-45ed-9863-a23433c2072f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:51.810382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda131f5-9a42-45ed-9863-a23433c2072f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.810669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda131f5-9a42-45ed-9863-a23433c2072f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.810943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda131f5-9a42-45ed-9863-a23433c2072f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.811742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda131f5-9a42-45ed-9863-a23433c2072f None admin] RBAC: Adding query filter params (interface=public, service_id=011a34776f2241f39e622bf212e4fba4, region_id=RegionOne) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:51.812053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda131f5-9a42-45ed-9863-a23433c2072f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:51.813436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda131f5-9a42-45ed-9863-a23433c2072f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:51.818789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 486/995] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jun 16 00:17:51 2022] GET /identity/v3/endpoints?service_id=011a34776f2241f39e622bf212e4fba4&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:51.824594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:51.926179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158e86e-62a6-4e19-b9a0-788b1813b25c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:51.959333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f32397-3aa5-4ffc-beef-46163d922164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:51.959999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32397-3aa5-4ffc-beef-46163d922164 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:51.960844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32397-3aa5-4ffc-beef-46163d922164 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:51.961122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32397-3aa5-4ffc-beef-46163d922164 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:51.961797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f32397-3aa5-4ffc-beef-46163d922164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:51.963140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f32397-3aa5-4ffc-beef-46163d922164 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:51.969543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 510/996] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:51 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:17:52.819585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8a5d9-7d3b-4598-9ed7-96c62702aade None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:52.820812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8a5d9-7d3b-4598-9ed7-96c62702aade None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:52.820812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8a5d9-7d3b-4598-9ed7-96c62702aade None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:52.822051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 487/997] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:52.827274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a836b9-42db-4eb1-bd74-979f9c3af5a6 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:52.827667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a836b9-42db-4eb1-bd74-979f9c3af5a6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:52.827854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a836b9-42db-4eb1-bd74-979f9c3af5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:52.845042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e8a836b9-42db-4eb1-bd74-979f9c3af5a6 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:52.846165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e8a836b9-42db-4eb1-bd74-979f9c3af5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:52.856219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:52.912334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 511/998] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:52.936081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8a5d9-7d3b-4598-9ed7-96c62702aade None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:53.029537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:53.038815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8a5d9-7d3b-4598-9ed7-96c62702aade None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:53.073621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:53.074425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:53.074712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:53.074988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:53.075644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:53.077044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:53.089345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:53.090400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:17:53.091277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:53.092747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 488/999] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jun 16 00:17:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:53.096588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a836b9-42db-4eb1-bd74-979f9c3af5a6 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:53.201974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a836b9-42db-4eb1-bd74-979f9c3af5a6 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:53.236650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:53.237611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:53.238082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:53.238538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:53.239444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:53.239931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:53.241514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:53.251090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 512/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:53.252645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:53.360249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c188c-729d-4a40-a39d-03eb359e879f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:53.392851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed54da-bd12-4c96-b285-9e7ffc123922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:53.393516 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed54da-bd12-4c96-b285-9e7ffc123922 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:53.393807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed54da-bd12-4c96-b285-9e7ffc123922 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:53.394097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed54da-bd12-4c96-b285-9e7ffc123922 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:53.394806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed54da-bd12-4c96-b285-9e7ffc123922 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:53.395125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed54da-bd12-4c96-b285-9e7ffc123922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:53.396489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed54da-bd12-4c96-b285-9e7ffc123922 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:53.402009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 489/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jun 16 00:17:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:53.407475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:53.418890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:53.509488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9d87-4ea9-4163-b996-f0d9c5e02501 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:53.556121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5c4b2-9e9f-4469-9bcb-a3007ab382e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:53.556831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c4b2-9e9f-4469-9bcb-a3007ab382e4 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:53.557124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c4b2-9e9f-4469-9bcb-a3007ab382e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:53.557410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c4b2-9e9f-4469-9bcb-a3007ab382e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:53.558186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5c4b2-9e9f-4469-9bcb-a3007ab382e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:53.559597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5c4b2-9e9f-4469-9bcb-a3007ab382e4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:53.631743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 513/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:17:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:53.634838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845b0f1-9f9a-4416-aeed-7ff88eb3f68f None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:53.635134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845b0f1-9f9a-4416-aeed-7ff88eb3f68f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:53.635416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845b0f1-9f9a-4416-aeed-7ff88eb3f68f None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:53.636993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 490/1003] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:53.642519 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4898-8bf1-46b6-916e-b2e541a7c8e0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:53.642859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4898-8bf1-46b6-916e-b2e541a7c8e0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:53.643147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4898-8bf1-46b6-916e-b2e541a7c8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:53.661137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-766c4898-8bf1-46b6-916e-b2e541a7c8e0 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:53.662587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-766c4898-8bf1-46b6-916e-b2e541a7c8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:53.732814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:53.746098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 514/1004] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:53.768959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845b0f1-9f9a-4416-aeed-7ff88eb3f68f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:53.796057 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:53.876433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845b0f1-9f9a-4416-aeed-7ff88eb3f68f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:53.911175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3cbd4-9680-4c0d-9ecb-844bdcd5d541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:53.911884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3cbd4-9680-4c0d-9ecb-844bdcd5d541 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:53.912171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3cbd4-9680-4c0d-9ecb-844bdcd5d541 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:53.912450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3cbd4-9680-4c0d-9ecb-844bdcd5d541 None admin] PATH_INFO: `/v3/services/image` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:53.913131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3cbd4-9680-4c0d-9ecb-844bdcd5d541 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:53.915032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3cbd4-9680-4c0d-9ecb-844bdcd5d541 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-7af3cbd4-9680-4c0d-9ecb-844bdcd5d541 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:53.926714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:53.927752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:53.929601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 491/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Jun 16 00:17:53 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:53.934557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c4898-8bf1-46b6-916e-b2e541a7c8e0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.037099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c4898-8bf1-46b6-916e-b2e541a7c8e0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:54.073162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:54.073829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.074113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.074392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.075095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] RBAC: Adding query filter params (name=image) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:54.075404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:54.076780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:54.081904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 515/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:54.082173 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8217d-4319-480e-b1e3-a651c5703151 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.082467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8217d-4319-480e-b1e3-a651c5703151 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.082807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8217d-4319-480e-b1e3-a651c5703151 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.084299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 492/1007] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:54.087716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.089752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81067b-83f8-4332-8b2f-bb126c389f9e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.090070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81067b-83f8-4332-8b2f-bb126c389f9e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.090309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81067b-83f8-4332-8b2f-bb126c389f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.107414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4c81067b-83f8-4332-8b2f-bb126c389f9e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:54.108632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4c81067b-83f8-4332-8b2f-bb126c389f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:54.150734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:54.180073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 493/1008] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:54.196094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b528f-330a-4237-98e1-9e4d6c01ff03 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:54.203658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81067b-83f8-4332-8b2f-bb126c389f9e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.221936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:54.235091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:54.235799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.236085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.236366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.237109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] RBAC: Adding query filter params (type=image) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:54.237423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:54.238928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:54.256552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 516/1009] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:54.322335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81067b-83f8-4332-8b2f-bb126c389f9e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:54.360895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:54.361721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.362014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.362297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] PATH_INFO: `/v3/services/compute` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.362971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:54.364368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:54.377079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:54.378184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:17:54.379075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:54.379881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 494/1010] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:54.388279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.489086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bdb60-3352-4320-8c6b-be41e07e3316 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:54.511611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:54.523321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:54.523992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.524279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.524639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.525346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:54.525658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:54.527014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:54.536183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 517/1011] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:54.537320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.626759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:54.645605 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a67f5-a614-4cd1-bd73-69f654acf64b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:54.681313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67769d25-d8b9-45b5-87c7-d8e2bf1c4fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:54.681983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769d25-d8b9-45b5-87c7-d8e2bf1c4fd3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.682271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769d25-d8b9-45b5-87c7-d8e2bf1c4fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.682547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769d25-d8b9-45b5-87c7-d8e2bf1c4fd3 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.683255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67769d25-d8b9-45b5-87c7-d8e2bf1c4fd3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:54.683563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67769d25-d8b9-45b5-87c7-d8e2bf1c4fd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:54.684959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67769d25-d8b9-45b5-87c7-d8e2bf1c4fd3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:54.690067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 495/1012] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jun 16 00:17:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:54.695541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.847296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fb936-4936-4ae3-b31e-e3f3c1702e57 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:54.880823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefbd17-f9fc-4678-80e4-7795fa29391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:54.881488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefbd17-f9fc-4678-80e4-7795fa29391d None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.881777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefbd17-f9fc-4678-80e4-7795fa29391d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.882111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefbd17-f9fc-4678-80e4-7795fa29391d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.882869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefbd17-f9fc-4678-80e4-7795fa29391d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:54.884227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefbd17-f9fc-4678-80e4-7795fa29391d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:54.963010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 518/1013] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:17:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:55.800473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5682c1-2a2b-42fa-bab3-1bc1bdffadb9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:55.801013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5682c1-2a2b-42fa-bab3-1bc1bdffadb9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:55.801124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5682c1-2a2b-42fa-bab3-1bc1bdffadb9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:55.802553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 496/1014] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:55.807666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2fe1a-ab7a-4781-8d58-f8b7c5e72d13 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:55.807960 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2fe1a-ab7a-4781-8d58-f8b7c5e72d13 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:55.808237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2fe1a-ab7a-4781-8d58-f8b7c5e72d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:55.825016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-11f2fe1a-ab7a-4781-8d58-f8b7c5e72d13 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:55.826178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-11f2fe1a-ab7a-4781-8d58-f8b7c5e72d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:55.861460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:55.901654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 519/1015] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:55 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:55.922510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5682c1-2a2b-42fa-bab3-1bc1bdffadb9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:56.024784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5682c1-2a2b-42fa-bab3-1bc1bdffadb9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:56.059287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8425f51-22ea-404e-856e-42b729d8bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:56.060080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8425f51-22ea-404e-856e-42b729d8bf3f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:56.060366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8425f51-22ea-404e-856e-42b729d8bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:56.060676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8425f51-22ea-404e-856e-42b729d8bf3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:56.061854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8425f51-22ea-404e-856e-42b729d8bf3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:56.063275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8425f51-22ea-404e-856e-42b729d8bf3f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:56.069536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 497/1016] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:56.071089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2fe1a-ab7a-4781-8d58-f8b7c5e72d13 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:56.172174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2fe1a-ab7a-4781-8d58-f8b7c5e72d13 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:56.206729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2c6f1-c0c9-4588-a489-5aea1219bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:56.207398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c6f1-c0c9-4588-a489-5aea1219bdda None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:56.207683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c6f1-c0c9-4588-a489-5aea1219bdda None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:56.207959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c6f1-c0c9-4588-a489-5aea1219bdda None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:56.208696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2c6f1-c0c9-4588-a489-5aea1219bdda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:56.210631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2c6f1-c0c9-4588-a489-5aea1219bdda None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:56.290170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 520/1017] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:56 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:56.399883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eab6d-0bac-4f6f-b342-fc7cde52d1d7 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:56.400217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eab6d-0bac-4f6f-b342-fc7cde52d1d7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:56.400452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eab6d-0bac-4f6f-b342-fc7cde52d1d7 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:56.402027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 498/1018] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:56.407382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a86d4-cd56-4cb4-961b-acbd340e693d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:56.407684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a86d4-cd56-4cb4-961b-acbd340e693d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:56.407972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a86d4-cd56-4cb4-961b-acbd340e693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:56.426439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-fc2a86d4-cd56-4cb4-961b-acbd340e693d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:56.427568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-fc2a86d4-cd56-4cb4-961b-acbd340e693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:56.443117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:56.498446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 521/1019] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:56 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:56.522506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eab6d-0bac-4f6f-b342-fc7cde52d1d7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:56.539060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:56.636447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eab6d-0bac-4f6f-b342-fc7cde52d1d7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:56.670934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aff692-ab6e-443a-8a49-14cad0eb7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:56.671606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aff692-ab6e-443a-8a49-14cad0eb7330 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:56.671888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aff692-ab6e-443a-8a49-14cad0eb7330 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:56.672162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aff692-ab6e-443a-8a49-14cad0eb7330 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:56.672931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aff692-ab6e-443a-8a49-14cad0eb7330 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:56.678857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aff692-ab6e-443a-8a49-14cad0eb7330 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:56.788558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 499/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:56 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:57.217170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43749e-a5b9-4354-86cf-dc8a20b0f027 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.217714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43749e-a5b9-4354-86cf-dc8a20b0f027 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.217714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43749e-a5b9-4354-86cf-dc8a20b0f027 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.219148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 522/1021] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:57.224186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a2110-1e9e-4831-86e5-3733179a9d05 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.224670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a2110-1e9e-4831-86e5-3733179a9d05 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.224882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a2110-1e9e-4831-86e5-3733179a9d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.242003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-728a2110-1e9e-4831-86e5-3733179a9d05 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:57.243193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-728a2110-1e9e-4831-86e5-3733179a9d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:57.315844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:57.323680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 500/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:57.346247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43749e-a5b9-4354-86cf-dc8a20b0f027 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:57.451309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43749e-a5b9-4354-86cf-dc8a20b0f027 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:57.492772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc473b6-2efe-478e-b94e-6e9297786f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:57.493930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc473b6-2efe-478e-b94e-6e9297786f40 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.494411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc473b6-2efe-478e-b94e-6e9297786f40 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.494882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc473b6-2efe-478e-b94e-6e9297786f40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.505857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc473b6-2efe-478e-b94e-6e9297786f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-2bc473b6-2efe-478e-b94e-6e9297786f40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:57.514407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:57.515354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:17:57.516226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:57.518076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 523/1023] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jun 16 00:17:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:57.523866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a2110-1e9e-4831-86e5-3733179a9d05 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:57.632177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a2110-1e9e-4831-86e5-3733179a9d05 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:57.666171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:57.666839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.667123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.667401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.668143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:57.668449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:57.669889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:57.683399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 501/1024] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jun 16 00:17:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:57.684332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc473b6-2efe-478e-b94e-6e9297786f40 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:57.697364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:57.783153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc473b6-2efe-478e-b94e-6e9297786f40 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:57.815625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:57.816404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.816714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.816991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.827782 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:57.829247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:57.845639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:57.846548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:57.852572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:57.852572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:57.852572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:57.852572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:57.852572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 524/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:57.852881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:57.915014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:57.960250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6c1f6-2d90-4ee0-9154-dbe25e4227a0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:57.993483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:57.994147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.994429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.994705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.995424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:57.995736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:57.997113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:58.020559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 502/1026] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:58.021793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:58.085109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:58.131588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f46238-d7c5-40d5-a603-f3789b0400dd None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:58.164686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:58.165479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:58.165762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:58.166038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] PATH_INFO: `/v3/users/nova` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:58.173502 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:58.175175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:58.189451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:58.190416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:58.191421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 525/1027] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:17:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:58.197908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:58.302051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8409c8-5eb7-4bb3-9faf-c5a392140065 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:58.322089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:58.336222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858abeb-83e1-4be6-bce8-204c017224dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:58.336898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858abeb-83e1-4be6-bce8-204c017224dc None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:58.337182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858abeb-83e1-4be6-bce8-204c017224dc None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:58.337459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858abeb-83e1-4be6-bce8-204c017224dc None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:58.338294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858abeb-83e1-4be6-bce8-204c017224dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:58.338604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858abeb-83e1-4be6-bce8-204c017224dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:58.340061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858abeb-83e1-4be6-bce8-204c017224dc None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:58.362217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 503/1028] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:17:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:58.363670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:58.466348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183542b-b224-4e4b-863a-e9d25a14409d None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:58.491535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:58.503487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592145-fe83-4135-8426-43e25ccbfc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:58.504320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592145-fe83-4135-8426-43e25ccbfc01 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:58.504779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592145-fe83-4135-8426-43e25ccbfc01 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:58.505228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592145-fe83-4135-8426-43e25ccbfc01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:58.528899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b40e-ff97-4f3f-be2b-4ce3cb459e10 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:58.529190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b40e-ff97-4f3f-be2b-4ce3cb459e10 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:58.529474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b40e-ff97-4f3f-be2b-4ce3cb459e10 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:58.530824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 504/1029] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:17:58.535842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac99038-88a2-4f9c-bf2d-dd355005fef7 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:58.536139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac99038-88a2-4f9c-bf2d-dd355005fef7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:58.536421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac99038-88a2-4f9c-bf2d-dd355005fef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:58.553736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-dac99038-88a2-4f9c-bf2d-dd355005fef7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:58.555424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-dac99038-88a2-4f9c-bf2d-dd355005fef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:58.582361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f592145-fe83-4135-8426-43e25ccbfc01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:58.583842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f592145-fe83-4135-8426-43e25ccbfc01 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:58.637355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 505/1030] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:58.662156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac99038-88a2-4f9c-bf2d-dd355005fef7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:58.691758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 526/1031] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:58.697946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592145-fe83-4135-8426-43e25ccbfc01 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:58.722771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:58.771995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac99038-88a2-4f9c-bf2d-dd355005fef7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:58.782890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:58.808976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592145-fe83-4135-8426-43e25ccbfc01 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:58.809284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a735d3-3af5-4eb2-92ee-e5b481e54710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:58.810011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a735d3-3af5-4eb2-92ee-e5b481e54710 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:58.810296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a735d3-3af5-4eb2-92ee-e5b481e54710 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:58.810574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a735d3-3af5-4eb2-92ee-e5b481e54710 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:58.811721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a735d3-3af5-4eb2-92ee-e5b481e54710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:58.813183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a735d3-3af5-4eb2-92ee-e5b481e54710 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:58.814792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 506/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:58.820906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a735d3-3af5-4eb2-92ee-e5b481e54710 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:58.850436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:58.851753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:58.852057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:58.852557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:58.862923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:58.864597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:58.883654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:58.884656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:58.885696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:58.885696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:58.885696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:58.885696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:58.885696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 527/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:58.890370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:58.929391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a735d3-3af5-4eb2-92ee-e5b481e54710 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:58.968825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f205c5-2a76-4372-a764-751b8f551d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:58.969535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f205c5-2a76-4372-a764-751b8f551d4a None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:58.969824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f205c5-2a76-4372-a764-751b8f551d4a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:58.970105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f205c5-2a76-4372-a764-751b8f551d4a None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:58.970819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f205c5-2a76-4372-a764-751b8f551d4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:58.972244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f205c5-2a76-4372-a764-751b8f551d4a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:58.974336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:58.999936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709518c4-1f92-4ce8-a545-e72ed7484ef1 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:59.032612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13db95-da11-444c-9a7f-ed62b90fb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:59.033274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13db95-da11-444c-9a7f-ed62b90fb685 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.033564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13db95-da11-444c-9a7f-ed62b90fb685 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.033848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13db95-da11-444c-9a7f-ed62b90fb685 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.034568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13db95-da11-444c-9a7f-ed62b90fb685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:59.034877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13db95-da11-444c-9a7f-ed62b90fb685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:59.036248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13db95-da11-444c-9a7f-ed62b90fb685 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:59.048212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 528/1034] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:59.049678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbbccd-6a4a-4ef3-abde-a900baf4a3e5 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.049973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbbccd-6a4a-4ef3-abde-a900baf4a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.050254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbbccd-6a4a-4ef3-abde-a900baf4a3e5 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.051548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 529/1035] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:59.054840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbbccd-6a4a-4ef3-abde-a900baf4a3e5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:59.064710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 507/1036] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:17:58 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:59.066908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911420f7-357c-41f1-b89f-47b7498e3faf None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.066908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911420f7-357c-41f1-b89f-47b7498e3faf None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.066908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911420f7-357c-41f1-b89f-47b7498e3faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.085914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-911420f7-357c-41f1-b89f-47b7498e3faf None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:59.087432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-911420f7-357c-41f1-b89f-47b7498e3faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:59.117877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:59.162152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 508/1037] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:59.188065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911420f7-357c-41f1-b89f-47b7498e3faf None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:59.201669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbbccd-6a4a-4ef3-abde-a900baf4a3e5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:59.248407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d38028-75e3-457c-b433-279d8280341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:59.249249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d38028-75e3-457c-b433-279d8280341f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.249533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d38028-75e3-457c-b433-279d8280341f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.249806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d38028-75e3-457c-b433-279d8280341f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.252898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:59.258264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d38028-75e3-457c-b433-279d8280341f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:59.263200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d38028-75e3-457c-b433-279d8280341f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-82d38028-75e3-457c-b433-279d8280341f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:59.274405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:59.275426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:17:59.276377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 530/1038] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:59.282261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d38028-75e3-457c-b433-279d8280341f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:59.310053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911420f7-357c-41f1-b89f-47b7498e3faf None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:59.347444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:59.348244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.348546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.348839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] PATH_INFO: `/v3/services/image` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.349500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:59.351439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:59.363310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:59.364622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:17:59.365689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 509/1039] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Jun 16 00:17:59 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:59.370429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:59.396638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d38028-75e3-457c-b433-279d8280341f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:59.400003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:59.434829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:59.435507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.435793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.436075 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.436898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:59.437215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:59.438645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:59.450464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 531/1040] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:59.456641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:59.479391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de392a-c53a-492b-912c-e3d34f8a064d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:59.514142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:59.514814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.515100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.515379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.516087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] RBAC: Adding query filter params (name=image) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:59.516398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:59.517784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:59.527745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 510/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:59.527899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:59.532338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:59.572386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2aa594-7729-49ec-9f91-2294d0dc3457 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:59.609079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:59.610068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.610580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.611064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.612112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] RBAC: Adding query filter params (role.id=aaf93b32d93946789fbde39f351cbc44, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=dea47e0ec737422c8f679e21174f8cab) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:59.612660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:59.614338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:59.620250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 532/1042] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:59 2022] GET /identity/v3/role_assignments?user.id=dea47e0ec737422c8f679e21174f8cab&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=aaf93b32d93946789fbde39f351cbc44 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:59.652770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:59.662794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06363457-e35a-4c3b-8659-50dbbe2859f0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:59.701467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:59.702156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.702439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.702715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.703444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] RBAC: Adding query filter params (type=image) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:59.703859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:59.705191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:59.715990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 511/1043] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:59.718349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:59.825698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1d35a-30a7-40d4-848f-e9accaa7f340 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:59.843791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:59.860161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc5a1d-3059-4ad8-9dfc-177227a2e1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:59.864360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc5a1d-3059-4ad8-9dfc-177227a2e1c1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:59.864360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc5a1d-3059-4ad8-9dfc-177227a2e1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:59.864360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc5a1d-3059-4ad8-9dfc-177227a2e1c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:59.864360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc5a1d-3059-4ad8-9dfc-177227a2e1c1 None admin] RBAC: Adding query filter params (interface=public, service_id=ddac30c9dc9e4c949097d8bb9d54dac1, region_id=RegionOne) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:59.864360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc5a1d-3059-4ad8-9dfc-177227a2e1c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:59.864360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc5a1d-3059-4ad8-9dfc-177227a2e1c1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:59.869058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 533/1044] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jun 16 00:17:59 2022] GET /identity/v3/endpoints?service_id=ddac30c9dc9e4c949097d8bb9d54dac1&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:59.874480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:59.976061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c234e-00d1-4ac8-a3d8-9ed0ddc2c9c8 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:00.009943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208beae-1112-49c3-bc68-bb1854c859c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:00.010608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208beae-1112-49c3-bc68-bb1854c859c1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:00.010893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208beae-1112-49c3-bc68-bb1854c859c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:00.011170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208beae-1112-49c3-bc68-bb1854c859c1 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:00.011849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208beae-1112-49c3-bc68-bb1854c859c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:00.013231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208beae-1112-49c3-bc68-bb1854c859c1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:00.028558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 512/1045] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:59 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:18:01.274570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a54d7-f5f6-4f7e-bc08-0406c0fec905 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:01.275162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a54d7-f5f6-4f7e-bc08-0406c0fec905 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:01.275326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a54d7-f5f6-4f7e-bc08-0406c0fec905 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:01.276651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 534/1046] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:01.282028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740d43f-e08a-40bc-b413-52254d3077b8 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:01.282467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740d43f-e08a-40bc-b413-52254d3077b8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:01.282547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740d43f-e08a-40bc-b413-52254d3077b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:01.300053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e740d43f-e08a-40bc-b413-52254d3077b8 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:01.301250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e740d43f-e08a-40bc-b413-52254d3077b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:01.315364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:01.368743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 513/1047] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:01.391001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a54d7-f5f6-4f7e-bc08-0406c0fec905 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:01.491811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a54d7-f5f6-4f7e-bc08-0406c0fec905 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:01.539812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f85bb9-be92-4db2-be1b-db6b580b3679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:01.540556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85bb9-be92-4db2-be1b-db6b580b3679 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:01.540838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85bb9-be92-4db2-be1b-db6b580b3679 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:01.541113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85bb9-be92-4db2-be1b-db6b580b3679 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:01.551460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f85bb9-be92-4db2-be1b-db6b580b3679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-c9f85bb9-be92-4db2-be1b-db6b580b3679 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:01.559593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:01.560589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:01.561472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 535/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:01.572823 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740d43f-e08a-40bc-b413-52254d3077b8 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:01.752954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740d43f-e08a-40bc-b413-52254d3077b8 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:01.754114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12483e84-a708-4a84-b7d2-3e19d0deb476 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:01.754409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12483e84-a708-4a84-b7d2-3e19d0deb476 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:01.754688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12483e84-a708-4a84-b7d2-3e19d0deb476 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:01.756209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 536/1049] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:18:01.761449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c5151-5888-4a38-a1ea-9c6357c0b225 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:01.761750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c5151-5888-4a38-a1ea-9c6357c0b225 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:01.762034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c5151-5888-4a38-a1ea-9c6357c0b225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:01.780393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-014c5151-5888-4a38-a1ea-9c6357c0b225 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:01.781622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-014c5151-5888-4a38-a1ea-9c6357c0b225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:01.801710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:01.802312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:01.803029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:01.803314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:01.804652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:01.804964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:01.806332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:01.819432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 514/1050] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:01.821467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:01.822297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:01.834597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:01.857778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 537/1051] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:18:01.880982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c5151-5888-4a38-a1ea-9c6357c0b225 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:01.924861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63705c-2968-4c3f-8fd8-5a616dac571d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:01.964553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2790c-7692-4c91-aca2-d98b73ca7c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:01.965974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2790c-7692-4c91-aca2-d98b73ca7c4a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:01.966266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2790c-7692-4c91-aca2-d98b73ca7c4a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:01.966543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2790c-7692-4c91-aca2-d98b73ca7c4a None admin] PATH_INFO: `/v3/users/7a14ba8dfd954499ae52608883abd7c4` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:01.981963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2790c-7692-4c91-aca2-d98b73ca7c4a None admin] RBAC: Authorizing `identity:get_user(user_id=7a14ba8dfd954499ae52608883abd7c4)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:01.983392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2790c-7692-4c91-aca2-d98b73ca7c4a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:01.984273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c5151-5888-4a38-a1ea-9c6357c0b225 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:01.989748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 515/1052] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:01 2022] GET /identity/v3/users/7a14ba8dfd954499ae52608883abd7c4 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:01.990668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2790c-7692-4c91-aca2-d98b73ca7c4a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.023675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a375b-4491-49f5-a6ec-72c7735f44a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.024639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a375b-4491-49f5-a6ec-72c7735f44a7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.024930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a375b-4491-49f5-a6ec-72c7735f44a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.025211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a375b-4491-49f5-a6ec-72c7735f44a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.035331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a375b-4491-49f5-a6ec-72c7735f44a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-888a375b-4491-49f5-a6ec-72c7735f44a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:02.043969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:02.045127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:18:02.046180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:02.050511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 538/1053] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jun 16 00:18:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:18:02.051353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a375b-4491-49f5-a6ec-72c7735f44a7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.060737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:02.084556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:02.096017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2790c-7692-4c91-aca2-d98b73ca7c4a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.133239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937a91a-1328-492f-b784-5ca406ee801c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.133868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a91a-1328-492f-b784-5ca406ee801c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.134150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a91a-1328-492f-b784-5ca406ee801c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.134435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a91a-1328-492f-b784-5ca406ee801c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.156801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a375b-4491-49f5-a6ec-72c7735f44a7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.194499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.195195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.195477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.195751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.196622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:02.196933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.198371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.207405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 539/1054] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:02.213962 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.232341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937a91a-1328-492f-b784-5ca406ee801c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.233830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937a91a-1328-492f-b784-5ca406ee801c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.301756 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:02.309470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:02.315997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23917bac-c4f6-47d3-9693-fd4eaba9436b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.334438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 516/1055] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:01 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:18:02.336305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef9d2c-a5fe-4bbd-86a5-cf52b7eb25d7 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.336624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef9d2c-a5fe-4bbd-86a5-cf52b7eb25d7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.336908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef9d2c-a5fe-4bbd-86a5-cf52b7eb25d7 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.338241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 517/1056] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:02.343068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef9d2c-a5fe-4bbd-86a5-cf52b7eb25d7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.349445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626208c-019e-4b50-a021-b4c73d4bb23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.350236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626208c-019e-4b50-a021-b4c73d4bb23e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.350521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626208c-019e-4b50-a021-b4c73d4bb23e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.351362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626208c-019e-4b50-a021-b4c73d4bb23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.361867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626208c-019e-4b50-a021-b4c73d4bb23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.363333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626208c-019e-4b50-a021-b4c73d4bb23e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-e626208c-019e-4b50-a021-b4c73d4bb23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:18:02.380710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:02.381838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:02.382864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:18:02.382864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:18:02.382864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:02.382864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:02.382864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 540/1057] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:18:02.383506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274e0e0-538a-4e53-a587-670ba30ea84e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.383763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274e0e0-538a-4e53-a587-670ba30ea84e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.384042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274e0e0-538a-4e53-a587-670ba30ea84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.400238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4274e0e0-538a-4e53-a587-670ba30ea84e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:02.401448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4274e0e0-538a-4e53-a587-670ba30ea84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:02.446975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef9d2c-a5fe-4bbd-86a5-cf52b7eb25d7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.466057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 541/1058] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:02.466895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274e0e0-538a-4e53-a587-670ba30ea84e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.491506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28665c28-12a4-477a-9cb2-5fd3fefa7723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.492350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28665c28-12a4-477a-9cb2-5fd3fefa7723 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.492661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28665c28-12a4-477a-9cb2-5fd3fefa7723 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.492936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28665c28-12a4-477a-9cb2-5fd3fefa7723 None admin] PATH_INFO: `/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.494108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28665c28-12a4-477a-9cb2-5fd3fefa7723 None admin] RBAC: Authorizing `identity:get_project(project_id=be4e5a0b51024ba6a3c3e11f8a11478a)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.495522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28665c28-12a4-477a-9cb2-5fd3fefa7723 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.497220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 518/1059] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:02.501355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28665c28-12a4-477a-9cb2-5fd3fefa7723 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.534975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:02.542538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:02.564770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274e0e0-538a-4e53-a587-670ba30ea84e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.598838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.599508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.599816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.600093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.600845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28665c28-12a4-477a-9cb2-5fd3fefa7723 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.600947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:02.601152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.602503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.613961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 542/1060] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:02.614770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.635765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.636579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.636872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.637152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] PATH_INFO: `/v3/services/image` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.637808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.639155 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:02.651145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:02.652361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:18:02.653335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:02.654332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 519/1061] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:02.655258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.717866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22c4b-9287-45cc-baec-e1829250a0c2 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.753114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.753765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.753935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:02.754060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.754315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.755189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=be4e5a0b51024ba6a3c3e11f8a11478a, user.id=7a14ba8dfd954499ae52608883abd7c4) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:02.755502 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.756900 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5965d-4d73-4c3b-be8d-65561b9664c0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.757037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.760193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:02.772614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 543/1062] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/role_assignments?user.id=7a14ba8dfd954499ae52608883abd7c4&scope.project.id=be4e5a0b51024ba6a3c3e11f8a11478a&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:02.772614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.800851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.801757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.802109 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.802624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.813802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.815377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:18:02.831705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:02.832927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:02.834112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 520/1063] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:02.841078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.882294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1c48-11f9-419e-a361-2b78d43f090a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.919250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.920410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.920730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.921010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.921728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] RBAC: Adding query filter params (name=image) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:02.922037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.923404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:02.929624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 544/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:02.935814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:02.951071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d120-8607-40cb-94d7-b949da5ad0c3 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:02.990584 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:02.991277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:02.991568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:02.991842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:02.992716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:02.993029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:02.994443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:03.011461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:03.011621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 521/1065] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:03.017008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:03.036464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:03.053331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6dedc-2cae-404d-b5fa-72675c59fc48 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:03.087972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:03.088829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:03.089219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:03.089599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:03.090409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] RBAC: Adding query filter params (type=image) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:03.090825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:03.092307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:03.098526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 545/1066] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:03.105392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:03.120364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d5b6-51b8-42e8-91e5-13804fb06bc6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:03.158266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:03.158976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:03.159258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:03.159558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:03.171257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:03.173152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:18:03.195534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:03.196724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:03.197849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:18:03.197849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:18:03.197849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:03.197849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:03.199220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 522/1067] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:18:03.204511 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:03.212584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675a2e-7aca-4a38-8610-526dc94d939a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:03.242525 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:03.254895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae155cbc-d57d-4a85-933f-1f34d0d72a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:03.257035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae155cbc-d57d-4a85-933f-1f34d0d72a87 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:03.257823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae155cbc-d57d-4a85-933f-1f34d0d72a87 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:03.258104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae155cbc-d57d-4a85-933f-1f34d0d72a87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:03.258852 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae155cbc-d57d-4a85-933f-1f34d0d72a87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:03.260223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae155cbc-d57d-4a85-933f-1f34d0d72a87 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:03.285400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:03.315692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3072b5e-0a6e-4da5-abf1-51207ca358ce None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:03.346716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 546/1068] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:18:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:18:03.357981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:03.358666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:03.358947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:03.359223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:03.359956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:03.360263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:03.362190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:03.377138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 523/1069] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:03.382994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae155cbc-d57d-4a85-933f-1f34d0d72a87 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:03.492509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae155cbc-d57d-4a85-933f-1f34d0d72a87 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:03.528326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:03.529161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:03.529447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:03.529724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:03.536517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:03.537960 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:18:03.553451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:03.555629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:03.557435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 547/1070] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:18:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:03.562660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:03.672409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eef10e-4334-4071-aaeb-47445918012a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:03.698242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:03.707362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9554ae-7332-4707-b9e0-2f79ac0ff719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:03.707989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9554ae-7332-4707-b9e0-2f79ac0ff719 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:03.708277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9554ae-7332-4707-b9e0-2f79ac0ff719 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:03.708597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9554ae-7332-4707-b9e0-2f79ac0ff719 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:03.709400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9554ae-7332-4707-b9e0-2f79ac0ff719 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:03.709713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9554ae-7332-4707-b9e0-2f79ac0ff719 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:03.711663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9554ae-7332-4707-b9e0-2f79ac0ff719 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:03.729110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 524/1071] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:18:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:03.730532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:03.783242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:03.838060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9deb7-aa53-4018-b65d-9e6acb03dfa1 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:03.874477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414b651-43a4-4e8a-b542-c767ff45b833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:03.875369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b651-43a4-4e8a-b542-c767ff45b833 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:03.875649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b651-43a4-4e8a-b542-c767ff45b833 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:03.875921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b651-43a4-4e8a-b542-c767ff45b833 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/dea47e0ec737422c8f679e21174f8cab/roles/aaf93b32d93946789fbde39f351cbc44` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:03.878102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414b651-43a4-4e8a-b542-c767ff45b833 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=dea47e0ec737422c8f679e21174f8cab, role_id=aaf93b32d93946789fbde39f351cbc44)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:03.879528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414b651-43a4-4e8a-b542-c767ff45b833 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:03.952155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 548/1072] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:18:03 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/dea47e0ec737422c8f679e21174f8cab/roles/aaf93b32d93946789fbde39f351cbc44 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:18:04.902679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eb605-3954-447e-94fa-5426b4869bb0 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:04.903300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eb605-3954-447e-94fa-5426b4869bb0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:04.903300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eb605-3954-447e-94fa-5426b4869bb0 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:04.904644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 525/1073] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:04.909849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b1acd-9242-4440-b197-3cb2aee69ca9 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:04.910280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b1acd-9242-4440-b197-3cb2aee69ca9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:04.910479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b1acd-9242-4440-b197-3cb2aee69ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:04.928056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-163b1acd-9242-4440-b197-3cb2aee69ca9 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:04.929808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-163b1acd-9242-4440-b197-3cb2aee69ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:04.994576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 549/1074] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:05.016720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0eb605-3954-447e-94fa-5426b4869bb0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:05.122868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0eb605-3954-447e-94fa-5426b4869bb0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:05.156684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb546-2a68-41a2-8d8d-4ac97ecd70e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:05.157812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb546-2a68-41a2-8d8d-4ac97ecd70e6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:05.158423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb546-2a68-41a2-8d8d-4ac97ecd70e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:05.158908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb546-2a68-41a2-8d8d-4ac97ecd70e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:05.170067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb546-2a68-41a2-8d8d-4ac97ecd70e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-f85eb546-2a68-41a2-8d8d-4ac97ecd70e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:05.178253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:05.179794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:05.180705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:05.186993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 526/1075] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:05.188244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b1acd-9242-4440-b197-3cb2aee69ca9 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:05.222839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:05.294046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b1acd-9242-4440-b197-3cb2aee69ca9 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:05.328568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:05.329636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:05.330162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:05.330647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:05.331602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:05.332126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:05.333715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:05.343330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 550/1076] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:05.348915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb546-2a68-41a2-8d8d-4ac97ecd70e6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:05.397117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:05.453005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb546-2a68-41a2-8d8d-4ac97ecd70e6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:05.489864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61c3ad-9f07-401b-b812-1ee670cc8cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:05.490745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61c3ad-9f07-401b-b812-1ee670cc8cc7 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:05.491029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61c3ad-9f07-401b-b812-1ee670cc8cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:05.491349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61c3ad-9f07-401b-b812-1ee670cc8cc7 None admin] PATH_INFO: `/v3/users/7a14ba8dfd954499ae52608883abd7c4` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:05.492567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61c3ad-9f07-401b-b812-1ee670cc8cc7 None admin] RBAC: Authorizing `identity:get_user(user_id=7a14ba8dfd954499ae52608883abd7c4)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:05.493985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61c3ad-9f07-401b-b812-1ee670cc8cc7 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:05.500255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 527/1077] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:05 2022] GET /identity/v3/users/7a14ba8dfd954499ae52608883abd7c4 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:05.501515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:05.519048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee5e7-5a7b-4dcd-a262-8b3370a08f0d None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:05.519393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee5e7-5a7b-4dcd-a262-8b3370a08f0d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:05.519615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee5e7-5a7b-4dcd-a262-8b3370a08f0d None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:05.521154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 528/1078] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:18:05.526626 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224231a5-a1d9-4fa0-b599-f312d0cb4c87 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:05.526920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224231a5-a1d9-4fa0-b599-f312d0cb4c87 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:05.527203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224231a5-a1d9-4fa0-b599-f312d0cb4c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:05.544652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-224231a5-a1d9-4fa0-b599-f312d0cb4c87 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:05.545845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-224231a5-a1d9-4fa0-b599-f312d0cb4c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:05.612780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:05.619404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda3780-bfa2-40ea-bd31-23a92d7bd0df None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:05.622302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 529/1079] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:05.644569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224231a5-a1d9-4fa0-b599-f312d0cb4c87 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:05.664676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d676b80-ab8f-452a-b7b1-20a06ee8576b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:05.665540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d676b80-ab8f-452a-b7b1-20a06ee8576b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:05.665823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d676b80-ab8f-452a-b7b1-20a06ee8576b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:05.666097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d676b80-ab8f-452a-b7b1-20a06ee8576b None admin] PATH_INFO: `/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:05.667456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d676b80-ab8f-452a-b7b1-20a06ee8576b None admin] RBAC: Authorizing `identity:get_project(project_id=be4e5a0b51024ba6a3c3e11f8a11478a)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:05.668747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d676b80-ab8f-452a-b7b1-20a06ee8576b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:05.670676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 551/1080] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:05 2022] GET /identity/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:05.676511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d676b80-ab8f-452a-b7b1-20a06ee8576b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:05.697364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:05.767466 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224231a5-a1d9-4fa0-b599-f312d0cb4c87 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:05.805689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfc7cf-c2bb-458f-ba6d-5394c0f76c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:05.806343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc7cf-c2bb-458f-ba6d-5394c0f76c91 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:05.806450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d676b80-ab8f-452a-b7b1-20a06ee8576b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:05.806634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc7cf-c2bb-458f-ba6d-5394c0f76c91 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:05.806918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc7cf-c2bb-458f-ba6d-5394c0f76c91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:05.843763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7957114-8b8c-4921-aaa2-a68c1b70167e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:05.844693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7957114-8b8c-4921-aaa2-a68c1b70167e None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:05.844984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7957114-8b8c-4921-aaa2-a68c1b70167e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:05.845264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7957114-8b8c-4921-aaa2-a68c1b70167e None admin] PATH_INFO: `/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a/users/7a14ba8dfd954499ae52608883abd7c4/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:05.847237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7957114-8b8c-4921-aaa2-a68c1b70167e None admin] RBAC: Authorizing `identity:create_grant(project_id=be4e5a0b51024ba6a3c3e11f8a11478a, user_id=7a14ba8dfd954499ae52608883abd7c4, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:05.848934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7957114-8b8c-4921-aaa2-a68c1b70167e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:05.879300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfc7cf-c2bb-458f-ba6d-5394c0f76c91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:05.880824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfc7cf-c2bb-458f-ba6d-5394c0f76c91 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:05.922733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 552/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:18:05 2022] PUT /identity/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a/users/7a14ba8dfd954499ae52608883abd7c4/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:18:05.925279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:05.991917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 530/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:18:05.995352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7957114-8b8c-4921-aaa2-a68c1b70167e None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:06.027405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:06.077053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582c22b-5b1d-44b0-98fd-ffd3b49a1980 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:06.077353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582c22b-5b1d-44b0-98fd-ffd3b49a1980 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:06.077623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582c22b-5b1d-44b0-98fd-ffd3b49a1980 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:06.079161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 531/1083] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:18:06.084715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804e5a4-78e9-4fc0-94ac-b9f1d5113107 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:06.085082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804e5a4-78e9-4fc0-94ac-b9f1d5113107 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:06.085313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804e5a4-78e9-4fc0-94ac-b9f1d5113107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:06.105151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-1804e5a4-78e9-4fc0-94ac-b9f1d5113107 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:06.106343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1804e5a4-78e9-4fc0-94ac-b9f1d5113107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:06.115881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7957114-8b8c-4921-aaa2-a68c1b70167e None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:06.155774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:06.156611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:06.156899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:06.157181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:06.168998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:06.170492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:06.181523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 532/1084] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:18:06.190348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:06.191281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:06.192252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:18:06.192252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:18:06.192252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:06.192252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:06.192252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 553/1085] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:06.198980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804e5a4-78e9-4fc0-94ac-b9f1d5113107 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:06.206428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:06.273788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:06.293303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:06.304771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804e5a4-78e9-4fc0-94ac-b9f1d5113107 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:06.318850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d25892-c132-4859-b400-7e5a324c6f1c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:06.339966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:06.340655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:06.340944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:06.341227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:06.341959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:06.342272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:06.343638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:06.355381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 533/1086] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:06.363343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aad9cb-269e-4d19-8174-954fa036c5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:06.364194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aad9cb-269e-4d19-8174-954fa036c5ff None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:06.364486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aad9cb-269e-4d19-8174-954fa036c5ff None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:06.364804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aad9cb-269e-4d19-8174-954fa036c5ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:06.377560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aad9cb-269e-4d19-8174-954fa036c5ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-93aad9cb-269e-4d19-8174-954fa036c5ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:06.386065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:06.387145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:06.388023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 554/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jun 16 00:18:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:18:06.394051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:06.503551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50545e02-80cf-49d9-8bae-cdc28fde6833 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:06.543523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:06.544115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:06.544401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:06.544732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:06.545478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:06.545785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:06.547135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:06.548504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:06.558465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 534/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jun 16 00:18:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:06.564505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aad9cb-269e-4d19-8174-954fa036c5ff None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:06.671652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aad9cb-269e-4d19-8174-954fa036c5ff None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:06.705468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:06.706267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:06.706553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:06.706861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:06.709572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:06.718169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:06.719629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:18:06.737024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:06.738135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:06.739085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:18:06.739085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:18:06.739085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:06.739085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:06.739085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 555/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:06.743992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:06.840494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb30251-7bc3-4a6b-83da-aa356299f49a None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:06.873466 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:06.874111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:06.874397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:06.874675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:06.875395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:06.875702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:06.877097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:06.888118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 535/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:06.893658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:06.992086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a2f31-52a7-4e24-ab1d-365757e87a61 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:07.025499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:07.026307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:07.026595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:07.026872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:07.035031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:07.036465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:18:07.050404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:07.051327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:07.052290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 556/1091] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jun 16 00:18:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:07.057856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:07.112366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:07.154347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d5da0-2179-4649-adbc-152c3330eb16 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:07.187407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:07.188065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:07.188347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:07.188657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:07.189485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:07.189791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:07.191192 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:07.207450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 536/1092] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jun 16 00:18:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:07.213022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:07.261164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:07.310895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3839ae-29e8-452c-a301-b0d3e44f44ad None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:07.342240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49082344-e1d2-411b-a359-827314568a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:07.342871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49082344-e1d2-411b-a359-827314568a74 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:07.343152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49082344-e1d2-411b-a359-827314568a74 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:07.343430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49082344-e1d2-411b-a359-827314568a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:07.406984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49082344-e1d2-411b-a359-827314568a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:07.408350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49082344-e1d2-411b-a359-827314568a74 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:07.454437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:07.504020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 557/1093] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:07.509792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:07.607124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a5521-ab9c-4758-a4d9-115b3c1251ff None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:07.621452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:07.641080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:07.641869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:07.646896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:07.646896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:07.652234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:07.653692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:18:07.675494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:07.676621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:07.677748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:18:07.677748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:18:07.677748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:18:07.677748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:07.677748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 537/1094] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:07.678474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49082344-e1d2-411b-a359-827314568a74 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:07.775711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49082344-e1d2-411b-a359-827314568a74 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:07.809124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:07.809898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:07.810244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:07.810587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:07.811372 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:07.811749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:07.813192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:07.824532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 558/1095] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:07.830030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:07.927577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24ab98-9377-4133-93f6-1ce60f973dbd None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:07.961341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793365b-55b1-403e-8b4a-9be714efaac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:07.962132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793365b-55b1-403e-8b4a-9be714efaac8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:07.962420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793365b-55b1-403e-8b4a-9be714efaac8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:07.962702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793365b-55b1-403e-8b4a-9be714efaac8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:07.969242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793365b-55b1-403e-8b4a-9be714efaac8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:07.970665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793365b-55b1-403e-8b4a-9be714efaac8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-6793365b-55b1-403e-8b4a-9be714efaac8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:18:07.984563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:07.985498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:07.986503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 538/1096] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:18:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:18:07.992048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:08.004968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:08.069209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89762660-eb1e-4e0c-9206-8072daf150ec None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:08.069741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89762660-eb1e-4e0c-9206-8072daf150ec None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:08.069857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89762660-eb1e-4e0c-9206-8072daf150ec None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:08.071129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 539/1097] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:18:08.076071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82f76-cabf-4b4d-827f-5806dee10d2c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:08.076361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82f76-cabf-4b4d-827f-5806dee10d2c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:08.076683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82f76-cabf-4b4d-827f-5806dee10d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:08.093158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7df82f76-cabf-4b4d-827f-5806dee10d2c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:08.094357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7df82f76-cabf-4b4d-827f-5806dee10d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:08.095653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dec91-875a-48c3-b605-059767a8edf3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:08.114421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:08.128921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:08.129582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:08.129866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:08.130146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:08.130929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:08.131241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:08.132735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:08.144708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 559/1098] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:18:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:08.150597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:08.164816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 540/1099] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:08.189750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82f76-cabf-4b4d-827f-5806dee10d2c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:08.218537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:08.250959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6120c-8d08-4ca5-a889-8fd203df1910 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:08.284640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:08.285293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:08.285583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:08.285865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:08.286442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82f76-cabf-4b4d-827f-5806dee10d2c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:08.286701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] RBAC: Adding query filter params (role.id=aaf93b32d93946789fbde39f351cbc44, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=dea47e0ec737422c8f679e21174f8cab) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:08.287020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:08.289068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:08.296621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 560/1100] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:08 2022] GET /identity/v3/role_assignments?user.id=dea47e0ec737422c8f679e21174f8cab&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=aaf93b32d93946789fbde39f351cbc44 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:08.322013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d28775-a073-4684-9455-09b815fb4e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:08.322830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28775-a073-4684-9455-09b815fb4e83 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:08.323110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28775-a073-4684-9455-09b815fb4e83 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:08.323384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28775-a073-4684-9455-09b815fb4e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:08.335537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d28775-a073-4684-9455-09b815fb4e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-83d28775-a073-4684-9455-09b815fb4e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:08.343227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:08.344411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:08.345366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 541/1101] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:18:08.351365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:08.455070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024e718-5538-4c27-b931-27c73c87413a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:08.485786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:08.491040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:08.491796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:08.492080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:08.495848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:08.495848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:08.495848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:08.495848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:08.503701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 561/1102] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:08.509723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d28775-a073-4684-9455-09b815fb4e83 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:08.529044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:08.621730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d28775-a073-4684-9455-09b815fb4e83 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:08.654828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef97919-fa49-4606-9e7b-18f39dabd180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:08.655695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef97919-fa49-4606-9e7b-18f39dabd180 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:08.655979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef97919-fa49-4606-9e7b-18f39dabd180 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:08.656253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef97919-fa49-4606-9e7b-18f39dabd180 None admin] PATH_INFO: `/v3/users/7a14ba8dfd954499ae52608883abd7c4` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:08.657467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef97919-fa49-4606-9e7b-18f39dabd180 None admin] RBAC: Authorizing `identity:get_user(user_id=7a14ba8dfd954499ae52608883abd7c4)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:08.658893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef97919-fa49-4606-9e7b-18f39dabd180 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:08.660610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 542/1103] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:08 2022] GET /identity/v3/users/7a14ba8dfd954499ae52608883abd7c4 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:08.666977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:08.769740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46209-b77b-4dc6-a108-225b5a74ade5 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:08.806818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eab439-3ce6-4e58-852f-28962f81b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:08.807473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eab439-3ce6-4e58-852f-28962f81b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:08.807762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eab439-3ce6-4e58-852f-28962f81b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:08.808043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eab439-3ce6-4e58-852f-28962f81b6b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:08.868972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:08.878050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eab439-3ce6-4e58-852f-28962f81b6b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:08.879619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eab439-3ce6-4e58-852f-28962f81b6b7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:08.990223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 562/1104] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:08.996075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef97919-fa49-4606-9e7b-18f39dabd180 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:09.081430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:09.097153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef97919-fa49-4606-9e7b-18f39dabd180 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:09.131925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405238f4-a9e4-415c-a11d-279d02bb36ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:09.132814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405238f4-a9e4-415c-a11d-279d02bb36ca None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:09.133107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405238f4-a9e4-415c-a11d-279d02bb36ca None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:09.133393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405238f4-a9e4-415c-a11d-279d02bb36ca None admin] PATH_INFO: `/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:09.134589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405238f4-a9e4-415c-a11d-279d02bb36ca None admin] RBAC: Authorizing `identity:get_project(project_id=be4e5a0b51024ba6a3c3e11f8a11478a)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:09.136010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405238f4-a9e4-415c-a11d-279d02bb36ca None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:09.137748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 543/1105] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:08 2022] GET /identity/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:09.144044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eab439-3ce6-4e58-852f-28962f81b6b7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:09.237132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:09.244047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eab439-3ce6-4e58-852f-28962f81b6b7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:09.276652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00293c38-58a2-44dd-b937-80adabfeaa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:09.277300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00293c38-58a2-44dd-b937-80adabfeaa4b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:09.277585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00293c38-58a2-44dd-b937-80adabfeaa4b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:09.277866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00293c38-58a2-44dd-b937-80adabfeaa4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:09.278693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00293c38-58a2-44dd-b937-80adabfeaa4b None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=be4e5a0b51024ba6a3c3e11f8a11478a, user.id=7a14ba8dfd954499ae52608883abd7c4) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:09.279007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00293c38-58a2-44dd-b937-80adabfeaa4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:09.280416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00293c38-58a2-44dd-b937-80adabfeaa4b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:09.286147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 563/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:09 2022] GET /identity/v3/role_assignments?user.id=7a14ba8dfd954499ae52608883abd7c4&scope.project.id=be4e5a0b51024ba6a3c3e11f8a11478a&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:11.415171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6dfa-7a20-4612-aae6-0ea9d9b1b7e9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:11.415695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6dfa-7a20-4612-aae6-0ea9d9b1b7e9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:11.415784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6dfa-7a20-4612-aae6-0ea9d9b1b7e9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:11.417129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 544/1107] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:11.422538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4970ec-0d90-4fbf-811f-5eb59c4b46cc None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:11.422903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4970ec-0d90-4fbf-811f-5eb59c4b46cc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:11.423101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4970ec-0d90-4fbf-811f-5eb59c4b46cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:11.439541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2c4970ec-0d90-4fbf-811f-5eb59c4b46cc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:11.440729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2c4970ec-0d90-4fbf-811f-5eb59c4b46cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:11.505363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 564/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:11.530147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd6dfa-7a20-4612-aae6-0ea9d9b1b7e9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:11.628607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd6dfa-7a20-4612-aae6-0ea9d9b1b7e9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:11.659997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31279a2-07fd-471c-8109-6e6e666b6b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:11.660795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31279a2-07fd-471c-8109-6e6e666b6b9e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:11.661077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31279a2-07fd-471c-8109-6e6e666b6b9e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:11.661348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31279a2-07fd-471c-8109-6e6e666b6b9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:11.662495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31279a2-07fd-471c-8109-6e6e666b6b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:11.664419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31279a2-07fd-471c-8109-6e6e666b6b9e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:11.666065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 545/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:11.671877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4970ec-0d90-4fbf-811f-5eb59c4b46cc None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:11.756585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:11.768212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4970ec-0d90-4fbf-811f-5eb59c4b46cc None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:11.801745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c689b7-6a4c-4aa0-9436-05e1a72b250a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:11.802418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c689b7-6a4c-4aa0-9436-05e1a72b250a None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:11.802703 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c689b7-6a4c-4aa0-9436-05e1a72b250a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:11.803014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c689b7-6a4c-4aa0-9436-05e1a72b250a None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:11.804306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c689b7-6a4c-4aa0-9436-05e1a72b250a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:11.805747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c689b7-6a4c-4aa0-9436-05e1a72b250a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:11.922220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 565/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:18:11 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:18:14.097534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b364f9-3f93-4b6e-bb09-851fea0ae831 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:14.098061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b364f9-3f93-4b6e-bb09-851fea0ae831 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:14.098146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b364f9-3f93-4b6e-bb09-851fea0ae831 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:14.099439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 546/1111] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:14.105147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d91-78be-4e42-a123-9f8dd93e0847 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:14.105563 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d91-78be-4e42-a123-9f8dd93e0847 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:14.105718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d91-78be-4e42-a123-9f8dd93e0847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:14.122684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e86b5d91-78be-4e42-a123-9f8dd93e0847 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:14.123867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e86b5d91-78be-4e42-a123-9f8dd93e0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:14.188116 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 566/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:14.212476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b364f9-3f93-4b6e-bb09-851fea0ae831 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:14.233832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:14.310733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b364f9-3f93-4b6e-bb09-851fea0ae831 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:14.349394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad074-9605-4d4b-9c00-d381e1dfcbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:14.350176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad074-9605-4d4b-9c00-d381e1dfcbba None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:14.350490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad074-9605-4d4b-9c00-d381e1dfcbba None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:14.350772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad074-9605-4d4b-9c00-d381e1dfcbba None admin] PATH_INFO: `/v3/roles/96431666cb824a7087dbd9f0e30c0c80` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:14.352239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bad074-9605-4d4b-9c00-d381e1dfcbba None admin] RBAC: Authorizing `identity:get_role(role_id=96431666cb824a7087dbd9f0e30c0c80)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:14.353634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bad074-9605-4d4b-9c00-d381e1dfcbba None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:14.354989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 547/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:14 2022] GET /identity/v3/roles/96431666cb824a7087dbd9f0e30c0c80 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:14.360718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b5d91-78be-4e42-a123-9f8dd93e0847 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:14.432420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:14.456598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b5d91-78be-4e42-a123-9f8dd93e0847 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:14.489409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf31c1-81fe-4081-bb5c-867db31dbcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:14.490267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf31c1-81fe-4081-bb5c-867db31dbcf2 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:14.490551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf31c1-81fe-4081-bb5c-867db31dbcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:14.490827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf31c1-81fe-4081-bb5c-867db31dbcf2 None admin] PATH_INFO: `/v3/users/ab11545fecf549be89d878074d0cbd89` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:14.504753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf31c1-81fe-4081-bb5c-867db31dbcf2 None admin] RBAC: Authorizing `identity:get_user(user_id=ab11545fecf549be89d878074d0cbd89)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:14.506180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf31c1-81fe-4081-bb5c-867db31dbcf2 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:14.508417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 567/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:14 2022] GET /identity/v3/users/ab11545fecf549be89d878074d0cbd89 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:14.513947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad074-9605-4d4b-9c00-d381e1dfcbba None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:14.597323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:14.609174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad074-9605-4d4b-9c00-d381e1dfcbba None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:14.642449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ebb4d-bb12-4f48-9fce-f935f6e89baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:14.643080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ebb4d-bb12-4f48-9fce-f935f6e89baa None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:14.643361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ebb4d-bb12-4f48-9fce-f935f6e89baa None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:14.643634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ebb4d-bb12-4f48-9fce-f935f6e89baa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:14.707652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ebb4d-bb12-4f48-9fce-f935f6e89baa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:14.709104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ebb4d-bb12-4f48-9fce-f935f6e89baa None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:14.791489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:14.804399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 548/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:14.810398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf31c1-81fe-4081-bb5c-867db31dbcf2 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:14.905913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf31c1-81fe-4081-bb5c-867db31dbcf2 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:14.935127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:14.939691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93431001-9255-4334-b17b-bdcc5441405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:14.940559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93431001-9255-4334-b17b-bdcc5441405e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:14.940846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93431001-9255-4334-b17b-bdcc5441405e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:14.941193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93431001-9255-4334-b17b-bdcc5441405e None admin] PATH_INFO: `/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:14.942354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93431001-9255-4334-b17b-bdcc5441405e None admin] RBAC: Authorizing `identity:get_project(project_id=be4e5a0b51024ba6a3c3e11f8a11478a)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:14.946201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93431001-9255-4334-b17b-bdcc5441405e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:14.946201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 568/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:14 2022] GET /identity/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:14.951667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ebb4d-bb12-4f48-9fce-f935f6e89baa None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:15.047875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ebb4d-bb12-4f48-9fce-f935f6e89baa None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:15.080198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c7313-0bcf-4a42-9f2a-771914761cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:15.080863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c7313-0bcf-4a42-9f2a-771914761cef None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:15.081150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c7313-0bcf-4a42-9f2a-771914761cef None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:15.081428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c7313-0bcf-4a42-9f2a-771914761cef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:15.082255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c7313-0bcf-4a42-9f2a-771914761cef None admin] RBAC: Adding query filter params (role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=be4e5a0b51024ba6a3c3e11f8a11478a, user.id=ab11545fecf549be89d878074d0cbd89) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:15.082566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c7313-0bcf-4a42-9f2a-771914761cef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:15.083967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c7313-0bcf-4a42-9f2a-771914761cef None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:15.089916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 549/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:14 2022] GET /identity/v3/role_assignments?user.id=ab11545fecf549be89d878074d0cbd89&scope.project.id=be4e5a0b51024ba6a3c3e11f8a11478a&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:17.181754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599f904-4020-47f3-adaa-37f705c6803b None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:17.182342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599f904-4020-47f3-adaa-37f705c6803b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:17.182342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599f904-4020-47f3-adaa-37f705c6803b None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:17.183666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 569/1118] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:17.188820 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb3931-9cba-409f-9c96-7f3e1115833f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:17.189255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb3931-9cba-409f-9c96-7f3e1115833f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:17.189376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb3931-9cba-409f-9c96-7f3e1115833f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:17.205545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-37cb3931-9cba-409f-9c96-7f3e1115833f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:17.206703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-37cb3931-9cba-409f-9c96-7f3e1115833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:17.251705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:17.270879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 550/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:17.294778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599f904-4020-47f3-adaa-37f705c6803b None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:17.395898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599f904-4020-47f3-adaa-37f705c6803b None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:17.429638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb28175-3203-4ffa-be30-979f30b9d918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:17.430484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb28175-3203-4ffa-be30-979f30b9d918 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:17.430772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb28175-3203-4ffa-be30-979f30b9d918 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:17.431050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb28175-3203-4ffa-be30-979f30b9d918 None admin] PATH_INFO: `/v3/roles/96431666cb824a7087dbd9f0e30c0c80` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:17.432582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb28175-3203-4ffa-be30-979f30b9d918 None admin] RBAC: Authorizing `identity:get_role(role_id=96431666cb824a7087dbd9f0e30c0c80)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:17.433889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb28175-3203-4ffa-be30-979f30b9d918 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:17.435231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 570/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:17 2022] GET /identity/v3/roles/96431666cb824a7087dbd9f0e30c0c80 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:17.441282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb3931-9cba-409f-9c96-7f3e1115833f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:17.534463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb3931-9cba-409f-9c96-7f3e1115833f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:17.566847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaeda3-12b8-495e-9c6d-b3894280eb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:17.567697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaeda3-12b8-495e-9c6d-b3894280eb48 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:17.567979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaeda3-12b8-495e-9c6d-b3894280eb48 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:17.568253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaeda3-12b8-495e-9c6d-b3894280eb48 None admin] PATH_INFO: `/v3/users/ab11545fecf549be89d878074d0cbd89` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:17.569955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaeda3-12b8-495e-9c6d-b3894280eb48 None admin] RBAC: Authorizing `identity:get_user(user_id=ab11545fecf549be89d878074d0cbd89)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:17.571354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaeda3-12b8-495e-9c6d-b3894280eb48 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:17.572961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 551/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:17 2022] GET /identity/v3/users/ab11545fecf549be89d878074d0cbd89 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:17.578509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb28175-3203-4ffa-be30-979f30b9d918 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:17.642117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:17.674314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb28175-3203-4ffa-be30-979f30b9d918 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:17.706350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7430f-dbfb-4b7e-ae6c-d2b970fe2c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:17.707153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7430f-dbfb-4b7e-ae6c-d2b970fe2c4c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:17.707433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7430f-dbfb-4b7e-ae6c-d2b970fe2c4c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:17.707703 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7430f-dbfb-4b7e-ae6c-d2b970fe2c4c None admin] PATH_INFO: `/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:17.708878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7430f-dbfb-4b7e-ae6c-d2b970fe2c4c None admin] RBAC: Authorizing `identity:get_project(project_id=be4e5a0b51024ba6a3c3e11f8a11478a)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:17.710865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7430f-dbfb-4b7e-ae6c-d2b970fe2c4c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:17.712541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 571/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:17 2022] GET /identity/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:17.718544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaeda3-12b8-495e-9c6d-b3894280eb48 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:17.766646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:17.814845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaeda3-12b8-495e-9c6d-b3894280eb48 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:17.846439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf23c63-bec0-4ec9-8e69-efaf2e4e0b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:17.847307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23c63-bec0-4ec9-8e69-efaf2e4e0b6a None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:17.847587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23c63-bec0-4ec9-8e69-efaf2e4e0b6a None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:17.847862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23c63-bec0-4ec9-8e69-efaf2e4e0b6a None admin] PATH_INFO: `/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a/users/ab11545fecf549be89d878074d0cbd89/roles/96431666cb824a7087dbd9f0e30c0c80` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:17.850019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf23c63-bec0-4ec9-8e69-efaf2e4e0b6a None admin] RBAC: Authorizing `identity:create_grant(project_id=be4e5a0b51024ba6a3c3e11f8a11478a, user_id=ab11545fecf549be89d878074d0cbd89, role_id=96431666cb824a7087dbd9f0e30c0c80)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:17.851963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf23c63-bec0-4ec9-8e69-efaf2e4e0b6a None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:17.921103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 552/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:18:17 2022] PUT /identity/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a/users/ab11545fecf549be89d878074d0cbd89/roles/96431666cb824a7087dbd9f0e30c0c80 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:18:20.029163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ca8b6-4b5d-498a-927f-ce7717ac485f None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:20.029701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ca8b6-4b5d-498a-927f-ce7717ac485f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:20.029701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ca8b6-4b5d-498a-927f-ce7717ac485f None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:20.031071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 572/1124] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:20.036079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55471d3-5621-4fa5-9d17-69d6f70f6719 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:20.036492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55471d3-5621-4fa5-9d17-69d6f70f6719 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:20.036688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55471d3-5621-4fa5-9d17-69d6f70f6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:20.055154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f55471d3-5621-4fa5-9d17-69d6f70f6719 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:20.056328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f55471d3-5621-4fa5-9d17-69d6f70f6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:20.120572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 553/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:20.143402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ca8b6-4b5d-498a-927f-ce7717ac485f None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:20.247619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ca8b6-4b5d-498a-927f-ce7717ac485f None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:20.279641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbfada-cfc4-4bde-bb5c-be8286b8798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:20.280421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfada-cfc4-4bde-bb5c-be8286b8798f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:20.280733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfada-cfc4-4bde-bb5c-be8286b8798f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:20.281007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfada-cfc4-4bde-bb5c-be8286b8798f None admin] PATH_INFO: `/v3/roles/96431666cb824a7087dbd9f0e30c0c80` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:20.282436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbfada-cfc4-4bde-bb5c-be8286b8798f None admin] RBAC: Authorizing `identity:get_role(role_id=96431666cb824a7087dbd9f0e30c0c80)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:20.283795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbfada-cfc4-4bde-bb5c-be8286b8798f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:20.285170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 573/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:20 2022] GET /identity/v3/roles/96431666cb824a7087dbd9f0e30c0c80 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:20.291491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55471d3-5621-4fa5-9d17-69d6f70f6719 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:20.330984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:20.391160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55471d3-5621-4fa5-9d17-69d6f70f6719 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:20.423633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef7a20-a09a-40e2-9801-8f1c4e8181ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:20.424784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef7a20-a09a-40e2-9801-8f1c4e8181ec None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:20.425071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef7a20-a09a-40e2-9801-8f1c4e8181ec None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:20.425350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef7a20-a09a-40e2-9801-8f1c4e8181ec None admin] PATH_INFO: `/v3/users/ab11545fecf549be89d878074d0cbd89` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:20.426534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef7a20-a09a-40e2-9801-8f1c4e8181ec None admin] RBAC: Authorizing `identity:get_user(user_id=ab11545fecf549be89d878074d0cbd89)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:20.427946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef7a20-a09a-40e2-9801-8f1c4e8181ec None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:20.430121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 554/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:20 2022] GET /identity/v3/users/ab11545fecf549be89d878074d0cbd89 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:20.435853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbfada-cfc4-4bde-bb5c-be8286b8798f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:20.439558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:20.531737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbfada-cfc4-4bde-bb5c-be8286b8798f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:20.563569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b07c7f-cfe5-4dd2-95ad-49674d9603ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:20.564203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b07c7f-cfe5-4dd2-95ad-49674d9603ed None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:20.564486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b07c7f-cfe5-4dd2-95ad-49674d9603ed None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:20.564790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b07c7f-cfe5-4dd2-95ad-49674d9603ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:20.630027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b07c7f-cfe5-4dd2-95ad-49674d9603ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:20.631486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b07c7f-cfe5-4dd2-95ad-49674d9603ed None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:20.634156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:20.724509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 574/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:20.730959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef7a20-a09a-40e2-9801-8f1c4e8181ec None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:20.827651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef7a20-a09a-40e2-9801-8f1c4e8181ec None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:20.836397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:20.861213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5496965-40b1-494c-857c-6c63b119fdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:20.862314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5496965-40b1-494c-857c-6c63b119fdee None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:20.862788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5496965-40b1-494c-857c-6c63b119fdee None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:20.863339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5496965-40b1-494c-857c-6c63b119fdee None admin] PATH_INFO: `/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:20.864546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5496965-40b1-494c-857c-6c63b119fdee None admin] RBAC: Authorizing `identity:get_project(project_id=be4e5a0b51024ba6a3c3e11f8a11478a)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:20.866505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5496965-40b1-494c-857c-6c63b119fdee None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:20.868194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 555/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:20 2022] GET /identity/v3/projects/be4e5a0b51024ba6a3c3e11f8a11478a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:20.874405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b07c7f-cfe5-4dd2-95ad-49674d9603ed None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:20.968469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b07c7f-cfe5-4dd2-95ad-49674d9603ed None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:20.970741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:21.001728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a57db-11ef-4c58-bb80-460ba67d6168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:21.002369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a57db-11ef-4c58-bb80-460ba67d6168 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:21.002655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a57db-11ef-4c58-bb80-460ba67d6168 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:21.002935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a57db-11ef-4c58-bb80-460ba67d6168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:21.003758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a57db-11ef-4c58-bb80-460ba67d6168 None admin] RBAC: Adding query filter params (role.id=96431666cb824a7087dbd9f0e30c0c80, scope.project.id=be4e5a0b51024ba6a3c3e11f8a11478a, user.id=ab11545fecf549be89d878074d0cbd89) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:21.004070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a57db-11ef-4c58-bb80-460ba67d6168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:21.005510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a57db-11ef-4c58-bb80-460ba67d6168 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:21.011223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 575/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:20 2022] GET /identity/v3/role_assignments?user.id=ab11545fecf549be89d878074d0cbd89&scope.project.id=be4e5a0b51024ba6a3c3e11f8a11478a&role.id=96431666cb824a7087dbd9f0e30c0c80 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:23.125778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38627f6-3061-4b6d-8a58-0b52336386c9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:23.126330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38627f6-3061-4b6d-8a58-0b52336386c9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:23.126330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38627f6-3061-4b6d-8a58-0b52336386c9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:23.127689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 556/1131] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:23.132878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f50f8d-bc4f-42f6-bafb-5fce4cd29a9f None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:23.133290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f50f8d-bc4f-42f6-bafb-5fce4cd29a9f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:23.133444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f50f8d-bc4f-42f6-bafb-5fce4cd29a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:23.149680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-46f50f8d-bc4f-42f6-bafb-5fce4cd29a9f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:23.150854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-46f50f8d-bc4f-42f6-bafb-5fce4cd29a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:23.214553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 576/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:23.237934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38627f6-3061-4b6d-8a58-0b52336386c9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:23.334207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38627f6-3061-4b6d-8a58-0b52336386c9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:23.366729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0d772-4a63-4069-a910-89e126811e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:23.367490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d772-4a63-4069-a910-89e126811e65 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:23.367771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d772-4a63-4069-a910-89e126811e65 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:23.368045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d772-4a63-4069-a910-89e126811e65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:23.369201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0d772-4a63-4069-a910-89e126811e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:23.370607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0d772-4a63-4069-a910-89e126811e65 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:23.372162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 557/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:23.377712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f50f8d-bc4f-42f6-bafb-5fce4cd29a9f None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:23.469347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:23.473118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f50f8d-bc4f-42f6-bafb-5fce4cd29a9f None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:23.506251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745c8ba-3c1f-4308-81a9-86fcd4a7b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:23.506931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745c8ba-3c1f-4308-81a9-86fcd4a7b613 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:23.507216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745c8ba-3c1f-4308-81a9-86fcd4a7b613 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:23.507495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745c8ba-3c1f-4308-81a9-86fcd4a7b613 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:23.508207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745c8ba-3c1f-4308-81a9-86fcd4a7b613 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:23.509701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745c8ba-3c1f-4308-81a9-86fcd4a7b613 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:23.591475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 577/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:18:23 2022] POST /identity/v3/projects => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:18:25.718884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a555a5-d8c3-4422-afe6-317c7d55b536 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:25.719420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a555a5-d8c3-4422-afe6-317c7d55b536 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:25.719420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a555a5-d8c3-4422-afe6-317c7d55b536 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:25.720811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 558/1135] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:25.725914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c3e80-7d1d-4aeb-897e-9ac7948068b7 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:25.726306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c3e80-7d1d-4aeb-897e-9ac7948068b7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:25.726487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c3e80-7d1d-4aeb-897e-9ac7948068b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:25.743721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-503c3e80-7d1d-4aeb-897e-9ac7948068b7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:25.744929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-503c3e80-7d1d-4aeb-897e-9ac7948068b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:25.808876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 578/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:18:25.833808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a555a5-d8c3-4422-afe6-317c7d55b536 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:25.879421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:25.932806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a555a5-d8c3-4422-afe6-317c7d55b536 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:25.964623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f817ed-0587-452c-9ecb-90e68026682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:25.965401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f817ed-0587-452c-9ecb-90e68026682f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:25.965685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f817ed-0587-452c-9ecb-90e68026682f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:25.965963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f817ed-0587-452c-9ecb-90e68026682f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:25.967124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f817ed-0587-452c-9ecb-90e68026682f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:25.968561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f817ed-0587-452c-9ecb-90e68026682f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:25.970143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 559/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:18:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:25.976192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c3e80-7d1d-4aeb-897e-9ac7948068b7 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:26.052851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:26.071559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c3e80-7d1d-4aeb-897e-9ac7948068b7 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:26.105313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31288d-d641-49e1-af16-7177cc8e1178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:26.105969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31288d-d641-49e1-af16-7177cc8e1178 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:26.106255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31288d-d641-49e1-af16-7177cc8e1178 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:26.106533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31288d-d641-49e1-af16-7177cc8e1178 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:26.107240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31288d-d641-49e1-af16-7177cc8e1178 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:26.108674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31288d-d641-49e1-af16-7177cc8e1178 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:26.199149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 579/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:18:25 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:18:28.340781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeddf15-9496-419b-826c-bcfdc8ebab32 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:28.341339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeddf15-9496-419b-826c-bcfdc8ebab32 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:28.341339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeddf15-9496-419b-826c-bcfdc8ebab32 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:28.342709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 560/1139] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:28.347706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500314b0-bbed-40a3-900b-ac73351b3c5d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:28.348125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500314b0-bbed-40a3-900b-ac73351b3c5d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:28.348269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500314b0-bbed-40a3-900b-ac73351b3c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:28.365096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-500314b0-bbed-40a3-900b-ac73351b3c5d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:28.366837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-500314b0-bbed-40a3-900b-ac73351b3c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:28.431124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 580/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:28.454187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeddf15-9496-419b-826c-bcfdc8ebab32 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:28.552289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeddf15-9496-419b-826c-bcfdc8ebab32 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:28.565364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:28.588498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761a6b4-b4c5-4cd9-8970-3c68f9e3c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:28.591308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761a6b4-b4c5-4cd9-8970-3c68f9e3c74d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:28.591669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761a6b4-b4c5-4cd9-8970-3c68f9e3c74d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:28.591996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761a6b4-b4c5-4cd9-8970-3c68f9e3c74d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:28.605316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761a6b4-b4c5-4cd9-8970-3c68f9e3c74d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-7761a6b4-b4c5-4cd9-8970-3c68f9e3c74d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:28.613020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:28.614249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:28.615395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 561/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:28.620926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500314b0-bbed-40a3-900b-ac73351b3c5d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:28.683920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:28.717307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500314b0-bbed-40a3-900b-ac73351b3c5d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:28.750296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:28.750954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:28.751239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:28.751519 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:28.752266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:28.752601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:28.753955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:28.762416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 581/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:28.768223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761a6b4-b4c5-4cd9-8970-3c68f9e3c74d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:28.866370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761a6b4-b4c5-4cd9-8970-3c68f9e3c74d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:28.898678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dde1d4-7058-44fc-bb11-a2e7d8cbda42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:28.899547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde1d4-7058-44fc-bb11-a2e7d8cbda42 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:28.899829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde1d4-7058-44fc-bb11-a2e7d8cbda42 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:28.900102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde1d4-7058-44fc-bb11-a2e7d8cbda42 None admin] PATH_INFO: `/v3/users/74d2cfff9352473796e9d6e3847249a2` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:28.913506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dde1d4-7058-44fc-bb11-a2e7d8cbda42 None admin] RBAC: Authorizing `identity:get_user(user_id=74d2cfff9352473796e9d6e3847249a2)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:28.914911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dde1d4-7058-44fc-bb11-a2e7d8cbda42 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:28.916576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 562/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:28 2022] GET /identity/v3/users/74d2cfff9352473796e9d6e3847249a2 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:28.922330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:29.018630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16097e4-57ee-496d-bffa-d4f3422bcbb8 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:29.043300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:29.050834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac325acf-7e18-4343-b4dc-61eac88041ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:29.053943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325acf-7e18-4343-b4dc-61eac88041ae None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:29.053943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325acf-7e18-4343-b4dc-61eac88041ae None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:29.053943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325acf-7e18-4343-b4dc-61eac88041ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:29.116602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac325acf-7e18-4343-b4dc-61eac88041ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:29.117971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac325acf-7e18-4343-b4dc-61eac88041ae None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:29.210349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 582/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:29.216126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dde1d4-7058-44fc-bb11-a2e7d8cbda42 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:29.245216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:29.310617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dde1d4-7058-44fc-bb11-a2e7d8cbda42 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:29.342592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f460463-32e3-428b-ba75-409e3d3717c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:29.343429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f460463-32e3-428b-ba75-409e3d3717c1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:29.343707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f460463-32e3-428b-ba75-409e3d3717c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:29.343979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f460463-32e3-428b-ba75-409e3d3717c1 None admin] PATH_INFO: `/v3/projects/cfe2b3af6808432eac7cbbc011b96748` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:29.345139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f460463-32e3-428b-ba75-409e3d3717c1 None admin] RBAC: Authorizing `identity:get_project(project_id=cfe2b3af6808432eac7cbbc011b96748)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:29.346550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f460463-32e3-428b-ba75-409e3d3717c1 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:29.348186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 563/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:29 2022] GET /identity/v3/projects/cfe2b3af6808432eac7cbbc011b96748 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:18:29.354637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac325acf-7e18-4343-b4dc-61eac88041ae None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:29.380960 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:29.448942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac325acf-7e18-4343-b4dc-61eac88041ae None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:29.481854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f795a-a419-4140-b356-66391912cb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:29.482488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f795a-a419-4140-b356-66391912cb5b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:29.482773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f795a-a419-4140-b356-66391912cb5b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:29.483050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f795a-a419-4140-b356-66391912cb5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:29.483875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f795a-a419-4140-b356-66391912cb5b None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=cfe2b3af6808432eac7cbbc011b96748, user.id=74d2cfff9352473796e9d6e3847249a2) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:29.484188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f795a-a419-4140-b356-66391912cb5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:29.485627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f795a-a419-4140-b356-66391912cb5b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:29.490920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 583/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:29 2022] GET /identity/v3/role_assignments?user.id=74d2cfff9352473796e9d6e3847249a2&scope.project.id=cfe2b3af6808432eac7cbbc011b96748&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:31.574270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8276c9-5034-4685-abd9-7ea6cbb6b6ab None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:31.574795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8276c9-5034-4685-abd9-7ea6cbb6b6ab None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:31.574905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8276c9-5034-4685-abd9-7ea6cbb6b6ab None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:31.576193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 564/1147] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:31.581758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d7d55-d50a-4b27-8c87-edf67ab80ea7 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:31.582166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d7d55-d50a-4b27-8c87-edf67ab80ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:31.582324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d7d55-d50a-4b27-8c87-edf67ab80ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:31.599181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-3e1d7d55-d50a-4b27-8c87-edf67ab80ea7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:31.600987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3e1d7d55-d50a-4b27-8c87-edf67ab80ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:31.664806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 584/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:31.687947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8276c9-5034-4685-abd9-7ea6cbb6b6ab None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:31.781503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:31.785971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8276c9-5034-4685-abd9-7ea6cbb6b6ab None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:31.818322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d9345-5652-4c7d-adcd-8fec60d567d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:31.819926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d9345-5652-4c7d-adcd-8fec60d567d1 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:31.820207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d9345-5652-4c7d-adcd-8fec60d567d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:31.820481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d9345-5652-4c7d-adcd-8fec60d567d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:31.830241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d9345-5652-4c7d-adcd-8fec60d567d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-ed0d9345-5652-4c7d-adcd-8fec60d567d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:31.838165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:31.839307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:31.840210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 565/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:31.845376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d7d55-d50a-4b27-8c87-edf67ab80ea7 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:31.868145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:31.946114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d7d55-d50a-4b27-8c87-edf67ab80ea7 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:31.979809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:31.980469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:31.980778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:31.981055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:31.981798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:31.982104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:31.983445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:31.991710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 585/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:31.997458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d9345-5652-4c7d-adcd-8fec60d567d1 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:32.092800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d9345-5652-4c7d-adcd-8fec60d567d1 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:32.125701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8cf45-3b1e-4e3a-a91c-9590bc0d893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:32.126548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8cf45-3b1e-4e3a-a91c-9590bc0d893b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:32.126832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8cf45-3b1e-4e3a-a91c-9590bc0d893b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:32.127109 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8cf45-3b1e-4e3a-a91c-9590bc0d893b None admin] PATH_INFO: `/v3/users/74d2cfff9352473796e9d6e3847249a2` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:32.128275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8cf45-3b1e-4e3a-a91c-9590bc0d893b None admin] RBAC: Authorizing `identity:get_user(user_id=74d2cfff9352473796e9d6e3847249a2)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:32.129720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8cf45-3b1e-4e3a-a91c-9590bc0d893b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:32.131325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 566/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:31 2022] GET /identity/v3/users/74d2cfff9352473796e9d6e3847249a2 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:32.136770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:32.219327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:32.235308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa9c3d-a6de-467d-9956-132a4a2f8a97 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:32.269701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a223636-6037-493c-910d-54f6e1ad88e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:32.270576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a223636-6037-493c-910d-54f6e1ad88e5 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:32.270860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a223636-6037-493c-910d-54f6e1ad88e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:32.271135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a223636-6037-493c-910d-54f6e1ad88e5 None admin] PATH_INFO: `/v3/projects/cfe2b3af6808432eac7cbbc011b96748` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:32.272830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a223636-6037-493c-910d-54f6e1ad88e5 None admin] RBAC: Authorizing `identity:get_project(project_id=cfe2b3af6808432eac7cbbc011b96748)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:32.274255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a223636-6037-493c-910d-54f6e1ad88e5 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:32.275928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 586/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:32 2022] GET /identity/v3/projects/cfe2b3af6808432eac7cbbc011b96748 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:32.288289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8cf45-3b1e-4e3a-a91c-9590bc0d893b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:32.315167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:32.383314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8cf45-3b1e-4e3a-a91c-9590bc0d893b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:32.418795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35698f2-d909-4dbb-86e0-f65dcea54efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:32.419681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35698f2-d909-4dbb-86e0-f65dcea54efb None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:32.419972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35698f2-d909-4dbb-86e0-f65dcea54efb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:32.420254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35698f2-d909-4dbb-86e0-f65dcea54efb None admin] PATH_INFO: `/v3/projects/cfe2b3af6808432eac7cbbc011b96748/users/74d2cfff9352473796e9d6e3847249a2/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:32.422221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35698f2-d909-4dbb-86e0-f65dcea54efb None admin] RBAC: Authorizing `identity:create_grant(project_id=cfe2b3af6808432eac7cbbc011b96748, user_id=74d2cfff9352473796e9d6e3847249a2, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:32.423657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35698f2-d909-4dbb-86e0-f65dcea54efb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:32.492343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 567/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:18:32 2022] PUT /identity/v3/projects/cfe2b3af6808432eac7cbbc011b96748/users/74d2cfff9352473796e9d6e3847249a2/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:18:34.564945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946cb7a-890d-4b77-8922-df622a068b76 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:34.565538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946cb7a-890d-4b77-8922-df622a068b76 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:34.565538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946cb7a-890d-4b77-8922-df622a068b76 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:34.566873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 587/1154] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:34.572986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b6216-dd7f-4fa6-8d3f-4c0f1053e7d4 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:34.573472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b6216-dd7f-4fa6-8d3f-4c0f1053e7d4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:34.573566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b6216-dd7f-4fa6-8d3f-4c0f1053e7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:34.594769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-974b6216-dd7f-4fa6-8d3f-4c0f1053e7d4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:34.595637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-974b6216-dd7f-4fa6-8d3f-4c0f1053e7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:34.659739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 568/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:34.683483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946cb7a-890d-4b77-8922-df622a068b76 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:34.781822 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946cb7a-890d-4b77-8922-df622a068b76 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:34.815279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba26cbb-c071-4a5c-a9de-322fc7e1a1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:34.815998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26cbb-c071-4a5c-a9de-322fc7e1a1fa None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:34.816282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26cbb-c071-4a5c-a9de-322fc7e1a1fa None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:34.816580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26cbb-c071-4a5c-a9de-322fc7e1a1fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:34.826981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba26cbb-c071-4a5c-a9de-322fc7e1a1fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-4ba26cbb-c071-4a5c-a9de-322fc7e1a1fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:34.834503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:34.835604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:34.836700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 588/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:34.841504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b6216-dd7f-4fa6-8d3f-4c0f1053e7d4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:34.858775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:34.937613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b6216-dd7f-4fa6-8d3f-4c0f1053e7d4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:34.969282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:34.969938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:34.970218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:34.970494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:34.971237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:34.971541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:34.972928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:34.981231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 569/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:34.987586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba26cbb-c071-4a5c-a9de-322fc7e1a1fa None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:35.003074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:35.084399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba26cbb-c071-4a5c-a9de-322fc7e1a1fa None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:35.120081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bbadd-0620-45aa-bed1-ef0a4f14eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:35.120966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bbadd-0620-45aa-bed1-ef0a4f14eda1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:35.121250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bbadd-0620-45aa-bed1-ef0a4f14eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:35.121522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bbadd-0620-45aa-bed1-ef0a4f14eda1 None admin] PATH_INFO: `/v3/users/74d2cfff9352473796e9d6e3847249a2` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:35.122673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bbadd-0620-45aa-bed1-ef0a4f14eda1 None admin] RBAC: Authorizing `identity:get_user(user_id=74d2cfff9352473796e9d6e3847249a2)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:35.124078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bbadd-0620-45aa-bed1-ef0a4f14eda1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:35.125707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 589/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:34 2022] GET /identity/v3/users/74d2cfff9352473796e9d6e3847249a2 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:35.131360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:35.205930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:35.227015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29561ba1-5301-4973-9d1b-7f84d9a9545d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:35.258675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0e6f-a2b5-454c-9039-f686c5ea0b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:35.259306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0e6f-a2b5-454c-9039-f686c5ea0b41 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:35.259590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0e6f-a2b5-454c-9039-f686c5ea0b41 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:35.259864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0e6f-a2b5-454c-9039-f686c5ea0b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:35.323368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0e6f-a2b5-454c-9039-f686c5ea0b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:35.324801 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0e6f-a2b5-454c-9039-f686c5ea0b41 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:35.398393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:35.419594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 570/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:35.425838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bbadd-0620-45aa-bed1-ef0a4f14eda1 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:35.502037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:35.519661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bbadd-0620-45aa-bed1-ef0a4f14eda1 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:35.552998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab440c-56c1-4cd2-a4b7-08bd4bb20cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:35.554346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab440c-56c1-4cd2-a4b7-08bd4bb20cdf None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:35.554624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab440c-56c1-4cd2-a4b7-08bd4bb20cdf None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:35.554895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab440c-56c1-4cd2-a4b7-08bd4bb20cdf None admin] PATH_INFO: `/v3/projects/cfe2b3af6808432eac7cbbc011b96748` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:35.556043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab440c-56c1-4cd2-a4b7-08bd4bb20cdf None admin] RBAC: Authorizing `identity:get_project(project_id=cfe2b3af6808432eac7cbbc011b96748)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:35.557473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab440c-56c1-4cd2-a4b7-08bd4bb20cdf None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:35.559120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 590/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:35 2022] GET /identity/v3/projects/cfe2b3af6808432eac7cbbc011b96748 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:35.565034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0e6f-a2b5-454c-9039-f686c5ea0b41 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:35.660906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0e6f-a2b5-454c-9039-f686c5ea0b41 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:35.694259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870548e-6020-486e-a83e-dbcf90b5b31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:35.694910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870548e-6020-486e-a83e-dbcf90b5b31c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:35.695199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870548e-6020-486e-a83e-dbcf90b5b31c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:35.695488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870548e-6020-486e-a83e-dbcf90b5b31c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:35.696317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870548e-6020-486e-a83e-dbcf90b5b31c None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=cfe2b3af6808432eac7cbbc011b96748, user.id=74d2cfff9352473796e9d6e3847249a2) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:35.696660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870548e-6020-486e-a83e-dbcf90b5b31c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:35.698072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870548e-6020-486e-a83e-dbcf90b5b31c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:35.703723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 571/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:35 2022] GET /identity/v3/role_assignments?user.id=74d2cfff9352473796e9d6e3847249a2&scope.project.id=cfe2b3af6808432eac7cbbc011b96748&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:37.821794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e31ef-b3c0-49a9-ba66-dde313daa0ad None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:37.822318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e31ef-b3c0-49a9-ba66-dde313daa0ad None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:37.822404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e31ef-b3c0-49a9-ba66-dde313daa0ad None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:37.823693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 591/1162] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:37.828809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a4f65-988f-4348-a99c-95c552b705ec None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:37.829206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a4f65-988f-4348-a99c-95c552b705ec None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:37.829376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a4f65-988f-4348-a99c-95c552b705ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:37.845680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-034a4f65-988f-4348-a99c-95c552b705ec None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:37.846796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-034a4f65-988f-4348-a99c-95c552b705ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:37.881742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:37.912226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 572/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:37.934570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e31ef-b3c0-49a9-ba66-dde313daa0ad None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:38.033440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e31ef-b3c0-49a9-ba66-dde313daa0ad None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:38.067033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec748d-bb80-46ad-bacd-b654bf214c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:38.067653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec748d-bb80-46ad-bacd-b654bf214c30 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:38.067940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec748d-bb80-46ad-bacd-b654bf214c30 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:38.068222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec748d-bb80-46ad-bacd-b654bf214c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:38.084943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:38.133414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec748d-bb80-46ad-bacd-b654bf214c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:38.134794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec748d-bb80-46ad-bacd-b654bf214c30 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:38.228299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 592/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:38.234413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a4f65-988f-4348-a99c-95c552b705ec None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:38.333915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a4f65-988f-4348-a99c-95c552b705ec None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:38.367293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246fb01-c48e-4e4b-9756-e7c2c0a80492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:38.368087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246fb01-c48e-4e4b-9756-e7c2c0a80492 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:38.368372 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246fb01-c48e-4e4b-9756-e7c2c0a80492 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:38.368679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246fb01-c48e-4e4b-9756-e7c2c0a80492 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:38.375134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246fb01-c48e-4e4b-9756-e7c2c0a80492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:38.376571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246fb01-c48e-4e4b-9756-e7c2c0a80492 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-8246fb01-c48e-4e4b-9756-e7c2c0a80492 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:38.390272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:38.391501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:38.392424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 573/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:18:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:38.398014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec748d-bb80-46ad-bacd-b654bf214c30 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:38.444971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:38.495136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec748d-bb80-46ad-bacd-b654bf214c30 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:38.526919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200145b8-cb0d-4a33-965e-224a6fff985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:38.527569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200145b8-cb0d-4a33-965e-224a6fff985a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:38.527853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200145b8-cb0d-4a33-965e-224a6fff985a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:38.528128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200145b8-cb0d-4a33-965e-224a6fff985a None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:38.528870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200145b8-cb0d-4a33-965e-224a6fff985a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:38.529176 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200145b8-cb0d-4a33-965e-224a6fff985a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:38.530535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200145b8-cb0d-4a33-965e-224a6fff985a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:38.537943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 593/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jun 16 00:18:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:40.644597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b6bdf-25ff-497b-b2b9-533ab47ed09a None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:40.645099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b6bdf-25ff-497b-b2b9-533ab47ed09a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:40.645186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b6bdf-25ff-497b-b2b9-533ab47ed09a None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:40.646507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 574/1167] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:40.651541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2c370-70d2-447a-9bd3-975c33eaedd6 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:40.651983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2c370-70d2-447a-9bd3-975c33eaedd6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:40.652112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2c370-70d2-447a-9bd3-975c33eaedd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:40.668420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f1e2c370-70d2-447a-9bd3-975c33eaedd6 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:40.669667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f1e2c370-70d2-447a-9bd3-975c33eaedd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:40.733941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 594/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:40.757002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b6bdf-25ff-497b-b2b9-533ab47ed09a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:40.787530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:40.855515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b6bdf-25ff-497b-b2b9-533ab47ed09a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:40.887643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58a15-39ce-457d-a0d7-d02fa36582fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:40.888286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58a15-39ce-457d-a0d7-d02fa36582fb None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:40.888598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58a15-39ce-457d-a0d7-d02fa36582fb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:40.888882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58a15-39ce-457d-a0d7-d02fa36582fb None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:40.889622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f58a15-39ce-457d-a0d7-d02fa36582fb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:40.891055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f58a15-39ce-457d-a0d7-d02fa36582fb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:40.955629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 575/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Thu Jun 16 00:18:40 2022] POST /identity/v3/domains => generated 267 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:18:43.060453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534bc60-392d-4c21-b721-35907ebb05a0 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:43.061026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534bc60-392d-4c21-b721-35907ebb05a0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:43.061026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534bc60-392d-4c21-b721-35907ebb05a0 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:43.062441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 595/1170] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:43.067437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c977a18-8e5f-4631-b33f-a796ec09cd07 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:43.067847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c977a18-8e5f-4631-b33f-a796ec09cd07 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:43.067997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c977a18-8e5f-4631-b33f-a796ec09cd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:43.085609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-3c977a18-8e5f-4631-b33f-a796ec09cd07 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:43.087044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3c977a18-8e5f-4631-b33f-a796ec09cd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:43.150918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 576/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:43.174538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534bc60-392d-4c21-b721-35907ebb05a0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:43.217552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:43.272719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534bc60-392d-4c21-b721-35907ebb05a0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:43.311812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cce798-cf7f-4011-a736-7e0dacc3f18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:43.312610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cce798-cf7f-4011-a736-7e0dacc3f18c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:43.312897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cce798-cf7f-4011-a736-7e0dacc3f18c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:43.313175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cce798-cf7f-4011-a736-7e0dacc3f18c None admin] PATH_INFO: `/v3/domains/4cf3420058444192bf06385fcfa69f2f` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:43.323288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cce798-cf7f-4011-a736-7e0dacc3f18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=4cf3420058444192bf06385fcfa69f2f)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:43.324741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cce798-cf7f-4011-a736-7e0dacc3f18c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:43.326368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 596/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Thu Jun 16 00:18:43 2022] GET /identity/v3/domains/4cf3420058444192bf06385fcfa69f2f => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:43.332314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c977a18-8e5f-4631-b33f-a796ec09cd07 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:43.355760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:43.429519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c977a18-8e5f-4631-b33f-a796ec09cd07 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:43.461903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e7ebe-b375-4435-891a-6a7a7166b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:43.462551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e7ebe-b375-4435-891a-6a7a7166b59f None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:43.462836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e7ebe-b375-4435-891a-6a7a7166b59f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:43.463116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e7ebe-b375-4435-891a-6a7a7166b59f None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:43.463831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e7ebe-b375-4435-891a-6a7a7166b59f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:43.465279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e7ebe-b375-4435-891a-6a7a7166b59f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:43.545142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 577/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:18:43 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:18:45.676387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46144-ecf0-4351-9162-e39b2289a53a None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:45.676927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46144-ecf0-4351-9162-e39b2289a53a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:45.677100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46144-ecf0-4351-9162-e39b2289a53a None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:45.678910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 597/1174] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:45.684129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a18dc8-2634-4181-b6b4-52d24bc4a5ea None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:45.684623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a18dc8-2634-4181-b6b4-52d24bc4a5ea None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:45.684762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a18dc8-2634-4181-b6b4-52d24bc4a5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:45.701212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-66a18dc8-2634-4181-b6b4-52d24bc4a5ea None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:45.702420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-66a18dc8-2634-4181-b6b4-52d24bc4a5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:45.767025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 578/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:18:45.792445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e46144-ecf0-4351-9162-e39b2289a53a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:45.889499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e46144-ecf0-4351-9162-e39b2289a53a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:45.890909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:45.922265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ebe5b-7920-4e5c-9905-1d7c4fa5f800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:45.923110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebe5b-7920-4e5c-9905-1d7c4fa5f800 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:45.923392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebe5b-7920-4e5c-9905-1d7c4fa5f800 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:45.923665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebe5b-7920-4e5c-9905-1d7c4fa5f800 None admin] PATH_INFO: `/v3/domains/4cf3420058444192bf06385fcfa69f2f` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:45.925097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ebe5b-7920-4e5c-9905-1d7c4fa5f800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4cf3420058444192bf06385fcfa69f2f)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:45.926553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ebe5b-7920-4e5c-9905-1d7c4fa5f800 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:45.928364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 598/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Thu Jun 16 00:18:45 2022] GET /identity/v3/domains/4cf3420058444192bf06385fcfa69f2f => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:45.934003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a18dc8-2634-4181-b6b4-52d24bc4a5ea None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:45.937023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:46.028896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a18dc8-2634-4181-b6b4-52d24bc4a5ea None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:46.061589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1e7f3-7052-4ca3-95f6-2ae06c2e5469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:46.062244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1e7f3-7052-4ca3-95f6-2ae06c2e5469 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:46.062527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1e7f3-7052-4ca3-95f6-2ae06c2e5469 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:46.062806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1e7f3-7052-4ca3-95f6-2ae06c2e5469 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:46.063508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1e7f3-7052-4ca3-95f6-2ae06c2e5469 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:46.064955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1e7f3-7052-4ca3-95f6-2ae06c2e5469 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:46.138100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 579/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:18:45 2022] POST /identity/v3/users => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:18:48.280584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b62ea-7ce5-48aa-bafb-8ad964cfccee None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:48.281171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b62ea-7ce5-48aa-bafb-8ad964cfccee None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:48.281171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b62ea-7ce5-48aa-bafb-8ad964cfccee None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:48.282491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 599/1178] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:48.287580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a7231-655e-4f63-b2b9-019ac311706e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:48.288004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a7231-655e-4f63-b2b9-019ac311706e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:48.288142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a7231-655e-4f63-b2b9-019ac311706e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:48.305215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-567a7231-655e-4f63-b2b9-019ac311706e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:48.306389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-567a7231-655e-4f63-b2b9-019ac311706e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:48.363737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:48.375007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 580/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:48.398229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b62ea-7ce5-48aa-bafb-8ad964cfccee None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:48.498631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b62ea-7ce5-48aa-bafb-8ad964cfccee None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:48.531313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f246-550b-4256-80e0-b0b007fabcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:48.532037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f246-550b-4256-80e0-b0b007fabcbb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:48.532326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f246-550b-4256-80e0-b0b007fabcbb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:48.532636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f246-550b-4256-80e0-b0b007fabcbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:48.543028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3f246-550b-4256-80e0-b0b007fabcbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-22e3f246-550b-4256-80e0-b0b007fabcbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:48.550881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:48.552070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:48.553800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 600/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:48.558558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a7231-655e-4f63-b2b9-019ac311706e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:48.653494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a7231-655e-4f63-b2b9-019ac311706e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:48.687103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:48.687761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:48.688044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:48.688323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:48.689099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:48.689406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:48.690769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:48.698647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 581/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:48.704383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f246-550b-4256-80e0-b0b007fabcbb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:48.796092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:48.863099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f246-550b-4256-80e0-b0b007fabcbb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:48.895039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c81f3-0e8f-4752-8fb5-119061123adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:48.895891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c81f3-0e8f-4752-8fb5-119061123adb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:48.896175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c81f3-0e8f-4752-8fb5-119061123adb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:48.897038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c81f3-0e8f-4752-8fb5-119061123adb None admin] PATH_INFO: `/v3/users/237188d6e75848e6bfa352094b8e7ff8` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:48.909475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c81f3-0e8f-4752-8fb5-119061123adb None admin] RBAC: Authorizing `identity:get_user(user_id=237188d6e75848e6bfa352094b8e7ff8)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:48.910900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c81f3-0e8f-4752-8fb5-119061123adb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:48.913102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 601/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:48 2022] GET /identity/v3/users/237188d6e75848e6bfa352094b8e7ff8 => generated 311 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:48.918865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:48.970757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:49.016220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b9955-3ea0-495f-8c3e-4976b0e27cab None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:49.048769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef944d-3509-43db-9211-f2cdb9892637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:49.049610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef944d-3509-43db-9211-f2cdb9892637 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:49.049903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef944d-3509-43db-9211-f2cdb9892637 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:49.050180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef944d-3509-43db-9211-f2cdb9892637 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:49.114009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef944d-3509-43db-9211-f2cdb9892637 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:49.115452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef944d-3509-43db-9211-f2cdb9892637 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:49.163811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:49.210042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 582/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:49.215905 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c81f3-0e8f-4752-8fb5-119061123adb None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:49.306849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:49.318173 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c81f3-0e8f-4752-8fb5-119061123adb None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:49.351396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d32e1-7ab1-478f-a734-da607827d98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:49.352263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d32e1-7ab1-478f-a734-da607827d98e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:49.352566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d32e1-7ab1-478f-a734-da607827d98e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:49.352847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d32e1-7ab1-478f-a734-da607827d98e None admin] PATH_INFO: `/v3/projects/c5f5a80da1004b95970c07d7fb7c1119` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:49.353987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d32e1-7ab1-478f-a734-da607827d98e None admin] RBAC: Authorizing `identity:get_project(project_id=c5f5a80da1004b95970c07d7fb7c1119)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:49.355384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d32e1-7ab1-478f-a734-da607827d98e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:49.357077 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 602/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:49 2022] GET /identity/v3/projects/c5f5a80da1004b95970c07d7fb7c1119 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:49.363471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef944d-3509-43db-9211-f2cdb9892637 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:49.458197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef944d-3509-43db-9211-f2cdb9892637 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:49.491727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cf2c0-e3ad-4291-8a93-c675a91c5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:49.492363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cf2c0-e3ad-4291-8a93-c675a91c5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:49.492678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cf2c0-e3ad-4291-8a93-c675a91c5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:49.492971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cf2c0-e3ad-4291-8a93-c675a91c5ef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:49.493783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cf2c0-e3ad-4291-8a93-c675a91c5ef2 None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=c5f5a80da1004b95970c07d7fb7c1119, user.id=237188d6e75848e6bfa352094b8e7ff8) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:49.494098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cf2c0-e3ad-4291-8a93-c675a91c5ef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:49.495507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cf2c0-e3ad-4291-8a93-c675a91c5ef2 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:49.500767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 583/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:49 2022] GET /identity/v3/role_assignments?user.id=237188d6e75848e6bfa352094b8e7ff8&scope.project.id=c5f5a80da1004b95970c07d7fb7c1119&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:51.592884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772317d2-a712-4252-b1aa-5a99de066dde None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:51.593432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772317d2-a712-4252-b1aa-5a99de066dde None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:51.593529 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772317d2-a712-4252-b1aa-5a99de066dde None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:51.595771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 603/1186] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:51.601684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb387-57e8-4804-8e17-f0eba05f1ac4 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:51.602117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb387-57e8-4804-8e17-f0eba05f1ac4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:51.602242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb387-57e8-4804-8e17-f0eba05f1ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:51.621684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-386eb387-57e8-4804-8e17-f0eba05f1ac4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:51.622876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-386eb387-57e8-4804-8e17-f0eba05f1ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:51.636610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:51.698958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 584/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:51.722215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772317d2-a712-4252-b1aa-5a99de066dde None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:51.821575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772317d2-a712-4252-b1aa-5a99de066dde None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:51.853000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdf86-274a-4e84-8aef-cc6d7b874ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:51.853710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdf86-274a-4e84-8aef-cc6d7b874ea0 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:51.853990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdf86-274a-4e84-8aef-cc6d7b874ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:51.854262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdf86-274a-4e84-8aef-cc6d7b874ea0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:51.863863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bdf86-274a-4e84-8aef-cc6d7b874ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-239bdf86-274a-4e84-8aef-cc6d7b874ea0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:51.871847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:51.872919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:18:51.873785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 604/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:51.879248 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eb387-57e8-4804-8e17-f0eba05f1ac4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:51.973753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eb387-57e8-4804-8e17-f0eba05f1ac4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:52.006438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:52.007089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:52.007374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:52.007651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:52.008390 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:52.008721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:52.010066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:52.017784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 585/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:52.023274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdf86-274a-4e84-8aef-cc6d7b874ea0 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:52.041094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:52.120658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdf86-274a-4e84-8aef-cc6d7b874ea0 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:52.152579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172320b-a827-48ac-9b8d-6135bb451bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:52.153425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172320b-a827-48ac-9b8d-6135bb451bd4 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:52.153705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172320b-a827-48ac-9b8d-6135bb451bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:52.153981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172320b-a827-48ac-9b8d-6135bb451bd4 None admin] PATH_INFO: `/v3/users/237188d6e75848e6bfa352094b8e7ff8` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:52.155132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172320b-a827-48ac-9b8d-6135bb451bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=237188d6e75848e6bfa352094b8e7ff8)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:52.156560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172320b-a827-48ac-9b8d-6135bb451bd4 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:52.158130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 605/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:52 2022] GET /identity/v3/users/237188d6e75848e6bfa352094b8e7ff8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:52.164131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:52.175372 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:52.261464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd001279-daaf-4da5-920d-925f9d539e64 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:52.293669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f0ee5-c649-48db-9e16-94a6d8f7ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:52.294518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0ee5-c649-48db-9e16-94a6d8f7ac16 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:52.294806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0ee5-c649-48db-9e16-94a6d8f7ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:52.295090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0ee5-c649-48db-9e16-94a6d8f7ac16 None admin] PATH_INFO: `/v3/projects/c5f5a80da1004b95970c07d7fb7c1119` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:52.296260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f0ee5-c649-48db-9e16-94a6d8f7ac16 None admin] RBAC: Authorizing `identity:get_project(project_id=c5f5a80da1004b95970c07d7fb7c1119)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:52.297713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f0ee5-c649-48db-9e16-94a6d8f7ac16 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:52.299641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 586/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:52 2022] GET /identity/v3/projects/c5f5a80da1004b95970c07d7fb7c1119 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:52.306106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172320b-a827-48ac-9b8d-6135bb451bd4 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:52.385635 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:52.400139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172320b-a827-48ac-9b8d-6135bb451bd4 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:52.433915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd60dc-7cf9-4bc1-a674-42120e4a2fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:52.434781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd60dc-7cf9-4bc1-a674-42120e4a2fd0 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:52.435060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd60dc-7cf9-4bc1-a674-42120e4a2fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:52.435331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd60dc-7cf9-4bc1-a674-42120e4a2fd0 None admin] PATH_INFO: `/v3/projects/c5f5a80da1004b95970c07d7fb7c1119/users/237188d6e75848e6bfa352094b8e7ff8/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:52.437249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd60dc-7cf9-4bc1-a674-42120e4a2fd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5f5a80da1004b95970c07d7fb7c1119, user_id=237188d6e75848e6bfa352094b8e7ff8, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:52.438668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd60dc-7cf9-4bc1-a674-42120e4a2fd0 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:52.506726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 606/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:18:52 2022] PUT /identity/v3/projects/c5f5a80da1004b95970c07d7fb7c1119/users/237188d6e75848e6bfa352094b8e7ff8/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:18:54.595200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b714-a6eb-4ea8-947a-6a9f26eb87f4 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:54.595594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b714-a6eb-4ea8-947a-6a9f26eb87f4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:54.595771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b714-a6eb-4ea8-947a-6a9f26eb87f4 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:54.597175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 587/1193] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:18:54.602958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8ee95-3724-4438-80a1-65a2726b6265 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:54.603391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8ee95-3724-4438-80a1-65a2726b6265 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:54.603503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8ee95-3724-4438-80a1-65a2726b6265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:54.621352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-78d8ee95-3724-4438-80a1-65a2726b6265 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:18:54.622542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-78d8ee95-3724-4438-80a1-65a2726b6265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:18:54.690098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 607/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:18:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:18:54.714509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614b714-a6eb-4ea8-947a-6a9f26eb87f4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:54.790174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:54.812726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614b714-a6eb-4ea8-947a-6a9f26eb87f4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:54.844875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e4e01-1d00-4e40-a2e6-a04a710b61d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:54.845588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e4e01-1d00-4e40-a2e6-a04a710b61d6 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:54.845869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e4e01-1d00-4e40-a2e6-a04a710b61d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:54.846146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e4e01-1d00-4e40-a2e6-a04a710b61d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:54.855858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e4e01-1d00-4e40-a2e6-a04a710b61d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-0c1e4e01-1d00-4e40-a2e6-a04a710b61d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:18:54.863287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:18:54.864427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:18:54.865878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 588/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:18:54.870816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8ee95-3724-4438-80a1-65a2726b6265 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:54.941953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:54.965992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8ee95-3724-4438-80a1-65a2726b6265 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:55.004959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:55.005689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:55.006532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:55.006860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:55.007851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:55.008333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:55.010186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:55.018437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 608/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:55.024407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e4e01-1d00-4e40-a2e6-a04a710b61d6 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:55.119316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e4e01-1d00-4e40-a2e6-a04a710b61d6 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:55.150305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752becb9-232b-454c-b81d-2a3fd013c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:55.151148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752becb9-232b-454c-b81d-2a3fd013c21d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:55.151427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752becb9-232b-454c-b81d-2a3fd013c21d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:55.151699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752becb9-232b-454c-b81d-2a3fd013c21d None admin] PATH_INFO: `/v3/users/237188d6e75848e6bfa352094b8e7ff8` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:55.152886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752becb9-232b-454c-b81d-2a3fd013c21d None admin] RBAC: Authorizing `identity:get_user(user_id=237188d6e75848e6bfa352094b8e7ff8)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:55.154296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752becb9-232b-454c-b81d-2a3fd013c21d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:55.155853 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 589/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jun 16 00:18:55 2022] GET /identity/v3/users/237188d6e75848e6bfa352094b8e7ff8 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:55.161330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:55.257366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f68f5d-6da1-4dd9-a4d2-ac113bb5359f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:55.289589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d0741-58f4-41cb-a861-d0b35a424d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:55.290226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d0741-58f4-41cb-a861-d0b35a424d28 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:55.290509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d0741-58f4-41cb-a861-d0b35a424d28 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:55.290785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d0741-58f4-41cb-a861-d0b35a424d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:55.294110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:55.355611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d0741-58f4-41cb-a861-d0b35a424d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:55.357084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d0741-58f4-41cb-a861-d0b35a424d28 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:55.451489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 609/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:18:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:18:55.457350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752becb9-232b-454c-b81d-2a3fd013c21d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:55.461526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:55.553969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752becb9-232b-454c-b81d-2a3fd013c21d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:55.586309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb1355-e8ef-42cd-bc9d-65091c1161c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:55.587185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb1355-e8ef-42cd-bc9d-65091c1161c7 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:55.587470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb1355-e8ef-42cd-bc9d-65091c1161c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:55.587749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb1355-e8ef-42cd-bc9d-65091c1161c7 None admin] PATH_INFO: `/v3/projects/c5f5a80da1004b95970c07d7fb7c1119` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:55.588957 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb1355-e8ef-42cd-bc9d-65091c1161c7 None admin] RBAC: Authorizing `identity:get_project(project_id=c5f5a80da1004b95970c07d7fb7c1119)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:55.590391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb1355-e8ef-42cd-bc9d-65091c1161c7 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:55.592085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 590/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jun 16 00:18:55 2022] GET /identity/v3/projects/c5f5a80da1004b95970c07d7fb7c1119 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:18:55.598172 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d0741-58f4-41cb-a861-d0b35a424d28 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:18:55.631212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:55.693102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d0741-58f4-41cb-a861-d0b35a424d28 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:18:55.726285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02134bd9-b0de-4a0e-8899-87e5b136076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:18:55.726869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02134bd9-b0de-4a0e-8899-87e5b136076b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:18:55.727155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02134bd9-b0de-4a0e-8899-87e5b136076b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:18:55.727436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02134bd9-b0de-4a0e-8899-87e5b136076b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:18:55.728259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02134bd9-b0de-4a0e-8899-87e5b136076b None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=c5f5a80da1004b95970c07d7fb7c1119, user.id=237188d6e75848e6bfa352094b8e7ff8) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:18:55.728598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02134bd9-b0de-4a0e-8899-87e5b136076b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:18:55.730020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02134bd9-b0de-4a0e-8899-87e5b136076b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:18:55.735649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 610/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:18:55 2022] GET /identity/v3/role_assignments?user.id=237188d6e75848e6bfa352094b8e7ff8&scope.project.id=c5f5a80da1004b95970c07d7fb7c1119&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:19:05.581679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=64 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:05.733102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=50 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:09.016887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1bf8f-0b33-402f-b17e-ed78eb164bad None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:09.017442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1bf8f-0b33-402f-b17e-ed78eb164bad None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:09.017442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1bf8f-0b33-402f-b17e-ed78eb164bad None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:09.018773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 591/1201] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:09.023811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105e1a4-0a23-4c94-8295-86190a8e0046 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:09.024255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105e1a4-0a23-4c94-8295-86190a8e0046 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:09.024430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105e1a4-0a23-4c94-8295-86190a8e0046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:09.041753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-0105e1a4-0a23-4c94-8295-86190a8e0046 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:09.042956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0105e1a4-0a23-4c94-8295-86190a8e0046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:09.107463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 611/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:20:09.132060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1bf8f-0b33-402f-b17e-ed78eb164bad None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:09.231515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1bf8f-0b33-402f-b17e-ed78eb164bad None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:09.264229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:09.265034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:09.265316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:09.265591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:09.275433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:09.277129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:09.295033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:09.296226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:09.297663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:09.297663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:09.297663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:09.297663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:09.297663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 592/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:09.302473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105e1a4-0a23-4c94-8295-86190a8e0046 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:09.399987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105e1a4-0a23-4c94-8295-86190a8e0046 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:09.428588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:09.434823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2107-d042-4660-a911-5ba9ce22d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:09.435488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2107-d042-4660-a911-5ba9ce22d105 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:09.435771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2107-d042-4660-a911-5ba9ce22d105 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:09.436048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2107-d042-4660-a911-5ba9ce22d105 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:09.436828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc2107-d042-4660-a911-5ba9ce22d105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:09.437140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc2107-d042-4660-a911-5ba9ce22d105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:09.438518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc2107-d042-4660-a911-5ba9ce22d105 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:09.449363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 612/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:09.454976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:09.506274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:09.552110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47323d-553a-43c6-90c5-3ecce1a78837 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:09.584578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dd3d2-3dae-4ab8-8eea-54457e497730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:09.585241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dd3d2-3dae-4ab8-8eea-54457e497730 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:09.585526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dd3d2-3dae-4ab8-8eea-54457e497730 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:09.585803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dd3d2-3dae-4ab8-8eea-54457e497730 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:09.586503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dd3d2-3dae-4ab8-8eea-54457e497730 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:09.587961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dd3d2-3dae-4ab8-8eea-54457e497730 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:09.685970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 593/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jun 16 00:20:09 2022] POST /identity/v3/users => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:20:11.844132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd29c72-bb19-4362-a050-c2ab4c2b71b1 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:11.844692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd29c72-bb19-4362-a050-c2ab4c2b71b1 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:11.844692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd29c72-bb19-4362-a050-c2ab4c2b71b1 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:11.846046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 613/1206] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:11.851267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772681b-eb19-4e79-ba8f-620f61efe5da None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:11.851677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772681b-eb19-4e79-ba8f-620f61efe5da None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:11.851830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772681b-eb19-4e79-ba8f-620f61efe5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:11.869090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8772681b-eb19-4e79-ba8f-620f61efe5da None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:11.870284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8772681b-eb19-4e79-ba8f-620f61efe5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:11.941735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 594/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:20:11.966910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd29c72-bb19-4362-a050-c2ab4c2b71b1 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:12.069592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd29c72-bb19-4362-a050-c2ab4c2b71b1 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:12.103718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd1545-ed4b-43a3-9067-06ca7fb3e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:12.104439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1545-ed4b-43a3-9067-06ca7fb3e54b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:12.104751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1545-ed4b-43a3-9067-06ca7fb3e54b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:12.105617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1545-ed4b-43a3-9067-06ca7fb3e54b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:12.116733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd1545-ed4b-43a3-9067-06ca7fb3e54b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-95bd1545-ed4b-43a3-9067-06ca7fb3e54b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:20:12.124470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:12.125647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:12.126879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 614/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:20:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:12.131822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772681b-eb19-4e79-ba8f-620f61efe5da None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:12.172238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:12.233000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772681b-eb19-4e79-ba8f-620f61efe5da None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:12.266770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:12.267440 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:12.267723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:12.268001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:12.268864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:12.269172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:12.270540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:12.278812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 595/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:20:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:12.292224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd1545-ed4b-43a3-9067-06ca7fb3e54b None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:12.345387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:12.407541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd1545-ed4b-43a3-9067-06ca7fb3e54b None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:12.442376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:12.443164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:12.443449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:12.443726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:12.454710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:12.456221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:12.474133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:12.475531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:12.477071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:12.477071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:12.477071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:12.477071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:12.477071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 615/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:12.481961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:12.582956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:12.586279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7b20b-0cae-4916-b9de-fde21fb1bdfe None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:12.620165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:12.620879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:12.621163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:12.621441 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:12.622158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:12.622465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:12.623816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:12.635056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 596/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:12.640795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:12.740735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675e19-585c-4896-a50f-c2957dda0aca None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:12.742194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:12.774058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:12.774857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:12.775136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:12.775411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:12.782782 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:12.784215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:20:12.798983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:12.802607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:12.803848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 616/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:12.806360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:12.908424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfd7e8-1ffe-413f-a79a-d2c522c3df3b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:12.941190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:12.941851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:12.942135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:12.942407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:12.943232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:12.943542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:12.944984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:12.962300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 597/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jun 16 00:20:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:12.968670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:13.067756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45991852-f5f3-4a84-a594-8d8eb49a1884 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:13.100143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543effa5-9c10-44c9-8c85-d56ba5106d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:13.100808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543effa5-9c10-44c9-8c85-d56ba5106d82 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:13.101094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543effa5-9c10-44c9-8c85-d56ba5106d82 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:13.101374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543effa5-9c10-44c9-8c85-d56ba5106d82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:13.131369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:13.167128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543effa5-9c10-44c9-8c85-d56ba5106d82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:13.169163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543effa5-9c10-44c9-8c85-d56ba5106d82 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:13.264966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 617/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:20:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:20:13.270896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:13.289916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:13.371776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590280ba-0c7f-4086-a17b-d88c4efa59a0 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:13.404447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:13.405260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:13.406195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:13.406195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:13.415621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:13.417096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:13.434985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:13.436071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:13.437038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:13.437038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:13.437038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:13.437038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:13.437038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 598/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:13.441942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543effa5-9c10-44c9-8c85-d56ba5106d82 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:13.501987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:13.539009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543effa5-9c10-44c9-8c85-d56ba5106d82 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:13.572561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:13.573222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:13.576229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:13.576229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:13.576229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:13.576229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:13.576229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:13.587807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 618/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:13.594393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:13.662560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:13.692148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ff52d-29e5-43f4-942c-28a2adc831aa None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:13.725399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:13.726183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:13.726467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:13.726743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:13.733543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:13.734977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:20:13.748744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:13.749885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:13.750985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 599/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:13.756143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:13.857911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f18fc-a446-4e19-b9df-7fd3a1fd4ff6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:13.886877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:13.898154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd3df6-c68e-4ab8-bef5-4e2f23faa3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:13.898877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd3df6-c68e-4ab8-bef5-4e2f23faa3bc None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:13.899165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd3df6-c68e-4ab8-bef5-4e2f23faa3bc None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:13.899444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd3df6-c68e-4ab8-bef5-4e2f23faa3bc None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:13.900230 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd3df6-c68e-4ab8-bef5-4e2f23faa3bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:13.900571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd3df6-c68e-4ab8-bef5-4e2f23faa3bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:13.902576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd3df6-c68e-4ab8-bef5-4e2f23faa3bc None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:13.913774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 619/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:20:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:13.919991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:14.017442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82768d-751d-4e7c-beb4-f52c70e88ee8 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:14.044830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:14.050660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f2d2d-73fe-40a5-9fbf-1a770aa3c333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:14.051292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f2d2d-73fe-40a5-9fbf-1a770aa3c333 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:14.051573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f2d2d-73fe-40a5-9fbf-1a770aa3c333 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:14.051844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f2d2d-73fe-40a5-9fbf-1a770aa3c333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:14.052701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f2d2d-73fe-40a5-9fbf-1a770aa3c333 None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=acc54fd2a9dd44b6b658c44edf170e83) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:14.053009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f2d2d-73fe-40a5-9fbf-1a770aa3c333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:14.054474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f2d2d-73fe-40a5-9fbf-1a770aa3c333 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:14.059693 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 600/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:20:13 2022] GET /identity/v3/role_assignments?user.id=acc54fd2a9dd44b6b658c44edf170e83&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:16.139535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1147947-805e-4811-8eb5-e7f64e0dabfc None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.140067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1147947-805e-4811-8eb5-e7f64e0dabfc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.140182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1147947-805e-4811-8eb5-e7f64e0dabfc None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.141532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 620/1220] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:16.147390 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea58ae-4822-4ce3-a62c-527fc9663c92 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.147936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea58ae-4822-4ce3-a62c-527fc9663c92 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.148044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea58ae-4822-4ce3-a62c-527fc9663c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.164677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-95ea58ae-4822-4ce3-a62c-527fc9663c92 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:16.165872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-95ea58ae-4822-4ce3-a62c-527fc9663c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:16.231921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 601/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:20:16.254846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1147947-805e-4811-8eb5-e7f64e0dabfc None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:16.355677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1147947-805e-4811-8eb5-e7f64e0dabfc None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:16.388050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61e5d2-37a5-4a0b-97de-122dd2d299f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:16.388859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61e5d2-37a5-4a0b-97de-122dd2d299f6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.389385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61e5d2-37a5-4a0b-97de-122dd2d299f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.389385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61e5d2-37a5-4a0b-97de-122dd2d299f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.399322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61e5d2-37a5-4a0b-97de-122dd2d299f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-0e61e5d2-37a5-4a0b-97de-122dd2d299f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:20:16.409001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:16.410220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:16.411282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 621/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:20:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:16.414906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea58ae-4822-4ce3-a62c-527fc9663c92 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:16.512785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea58ae-4822-4ce3-a62c-527fc9663c92 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:16.535688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:16.546583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:16.547246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.547593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.547870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.548645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] RBAC: Adding query filter params (name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:16.549093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:16.550687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:16.558358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 602/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:20:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:16.564073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61e5d2-37a5-4a0b-97de-122dd2d299f6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:16.609585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:16.665545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61e5d2-37a5-4a0b-97de-122dd2d299f6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:16.699967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:16.700786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.701076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.701358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.711517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:16.713008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:16.729488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:16.731082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:16.732280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:16.732280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:16.732280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:16.732280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:16.733120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 622/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:16.739370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:16.836380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5778-a230-49f7-a0e6-6c110f9cf539 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:16.870940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:16.871616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.872465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.872781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.873513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:16.873826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:16.875191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:16.887040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 603/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:20:16.892669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:16.984253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:16.990686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d34556-b8d1-4229-a161-09b178b74925 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:17.023250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:17.024043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:17.024334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:17.024653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:17.031950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:17.033416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:20:17.048190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:17.049215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:17.050688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 623/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:17.055349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:17.101110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:17.152322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44666cd8-ebda-424a-8acb-2da8fb4c4c48 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:17.186003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:17.186667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:17.186953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:17.187231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:17.188061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:17.188373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:17.189838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:17.206142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 604/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jun 16 00:20:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:17.211716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:17.310516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d169d54-2b57-4be4-bd48-ba9b638c96ab None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:17.343909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:17.344727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:17.345009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:17.345284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:17.355046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:17.357228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:17.375912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:17.377126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:17.378430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:17.378430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:17.378430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:17.378430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:17.378430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 624/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:17.383070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:17.480630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189cbbe-e40c-4b1c-9f73-82478b0c5444 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:17.483388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:17.514766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:17.515426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:17.515741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:17.516020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:17.516770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:17.517078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:17.518435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:17.529530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 605/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:17.535576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:17.545160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:17.632164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9117b-5769-489e-b658-5a4e7dbdfb55 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:17.665145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:17.665940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:17.666226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:17.666500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:17.673711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:17.675433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:20:17.689511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:17.690624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:17.691964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 625/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:17.696424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:17.795848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cdd4e-e4d9-421f-af76-176130bb72a3 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:17.828012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d55129-48a3-4a05-97e2-ed0c17124268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:17.828696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d55129-48a3-4a05-97e2-ed0c17124268 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:17.828979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d55129-48a3-4a05-97e2-ed0c17124268 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:17.829257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d55129-48a3-4a05-97e2-ed0c17124268 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:17.830032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d55129-48a3-4a05-97e2-ed0c17124268 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:17.830339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d55129-48a3-4a05-97e2-ed0c17124268 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:17.831747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d55129-48a3-4a05-97e2-ed0c17124268 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:17.843734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 606/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:17.856113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:17.914561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:17.947824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebec8fb-a70b-44eb-b226-35d05c4aa61c None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:17.979654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347ca5d-9dd3-4ea1-9941-a27ad7701969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:17.980660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ca5d-9dd3-4ea1-9941-a27ad7701969 None admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:17.980941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ca5d-9dd3-4ea1-9941-a27ad7701969 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:17.981769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ca5d-9dd3-4ea1-9941-a27ad7701969 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/acc54fd2a9dd44b6b658c44edf170e83/roles/78418cd7aaaf481e94926d572280f81e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:17.995904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347ca5d-9dd3-4ea1-9941-a27ad7701969 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=acc54fd2a9dd44b6b658c44edf170e83, role_id=78418cd7aaaf481e94926d572280f81e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:17.997377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347ca5d-9dd3-4ea1-9941-a27ad7701969 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:18.093558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 626/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:20:17 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/acc54fd2a9dd44b6b658c44edf170e83/roles/78418cd7aaaf481e94926d572280f81e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:20:20.200191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1a36-99cf-48d0-92fc-7ff0cf0c5dfd None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.200798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1a36-99cf-48d0-92fc-7ff0cf0c5dfd None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.200798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1a36-99cf-48d0-92fc-7ff0cf0c5dfd None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.202174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 607/1233] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:20.207327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faf02a-19c6-457f-92ea-1f90f9d6ebe1 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.207725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faf02a-19c6-457f-92ea-1f90f9d6ebe1 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.207888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faf02a-19c6-457f-92ea-1f90f9d6ebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.224640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e9faf02a-19c6-457f-92ea-1f90f9d6ebe1 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:20.225800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e9faf02a-19c6-457f-92ea-1f90f9d6ebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:20.289968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 627/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:20:20.313699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f1a36-99cf-48d0-92fc-7ff0cf0c5dfd None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:20.338505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:20.412533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f1a36-99cf-48d0-92fc-7ff0cf0c5dfd None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:20.444343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647faddd-12c5-409e-b09a-e0441158af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:20.445255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faddd-12c5-409e-b09a-e0441158af05 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.445537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faddd-12c5-409e-b09a-e0441158af05 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.445809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faddd-12c5-409e-b09a-e0441158af05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.456645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647faddd-12c5-409e-b09a-e0441158af05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-647faddd-12c5-409e-b09a-e0441158af05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:20:20.465003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:20.466851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:20.468135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 608/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jun 16 00:20:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:20.472795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9faf02a-19c6-457f-92ea-1f90f9d6ebe1 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:20.519235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:20.570529 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9faf02a-19c6-457f-92ea-1f90f9d6ebe1 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:20.604002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:20.605783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.605783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.605783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.606670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] RBAC: Adding query filter params (name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:20.606913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:20.608258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:20.616092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 628/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jun 16 00:20:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:20.622758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647faddd-12c5-409e-b09a-e0441158af05 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:20.709144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:20.724549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647faddd-12c5-409e-b09a-e0441158af05 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:20.756781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:20.757562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.757845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.758122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.768972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:20.775521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:20.787320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:20.788232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:20.789235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:20.789235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:20.789235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:20.789235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:20.789235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 609/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:20.794529 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:20.895070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a46a6-594d-4056-97fe-07d57270358e None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:20.903925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:20.927737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:20.928396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.928711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.928991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.929710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:20.930020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:20.931390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:20.943204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 629/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:20.949000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:21.045158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166e6d-5ab2-456c-becb-fe6398d17801 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:21.080151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:21.080969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.081252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.081530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.084258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:21.089028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:21.090678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:20:21.105910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:21.107686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:21.108982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 610/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:21.113472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:21.210589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ed119-912f-4e2b-9229-e96716793618 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:21.243739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:21.244402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.244766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.245047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.245879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:21.246190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:21.248246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:21.265182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 630/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jun 16 00:20:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:21.271012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:21.369686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7f745-1d0c-4b51-8864-df0cb5ccea61 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:21.402051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6add0b9-d7a0-4691-818d-d109ba129ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:21.402697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add0b9-d7a0-4691-818d-d109ba129ab5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.402978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add0b9-d7a0-4691-818d-d109ba129ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.403252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add0b9-d7a0-4691-818d-d109ba129ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.461547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:21.467858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6add0b9-d7a0-4691-818d-d109ba129ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:21.469325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6add0b9-d7a0-4691-818d-d109ba129ab5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:21.567030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 611/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:20:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:20:21.572868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:21.600375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:21.668428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a5459-a05e-4dba-a403-a5c4f2bac6c7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:21.701675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:21.702457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.702739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.703017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.713542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:21.715028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:21.733763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:21.735365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:21.736734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:21.736734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:21.736734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:21.736734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:21.736734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 631/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:21.740836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6add0b9-d7a0-4691-818d-d109ba129ab5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:21.838408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:21.844546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6add0b9-d7a0-4691-818d-d109ba129ab5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:21.877116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:21.877769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.878049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.878322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.879090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:21.879376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:21.880740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:21.892181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 612/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:21.898007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:21.976796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:21.998490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20478a5c-4c63-4ce1-9bb9-5c866aaa20d8 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:22.031180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:22.031957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:22.032241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:22.032514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:22.038991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:22.040457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:20:22.054729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:22.055793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:22.067679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 632/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:20:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:22.068334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:22.161811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee339cc8-1fe0-4c5b-8887-681b882068b5 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:22.195601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6bf0b-f8d8-4e3e-838e-e72cdb628502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:22.196258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6bf0b-f8d8-4e3e-838e-e72cdb628502 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:22.196568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6bf0b-f8d8-4e3e-838e-e72cdb628502 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:22.196852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6bf0b-f8d8-4e3e-838e-e72cdb628502 None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:22.197634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6bf0b-f8d8-4e3e-838e-e72cdb628502 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:22.197946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6bf0b-f8d8-4e3e-838e-e72cdb628502 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:22.199370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6bf0b-f8d8-4e3e-838e-e72cdb628502 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:22.210985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 613/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:22.217324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:22.315165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8723a-931d-4f8b-8387-d34434d2ba52 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:22.348568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23165c-f5b1-42bd-b7a9-a7f8045b5d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:22.349207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23165c-f5b1-42bd-b7a9-a7f8045b5d6d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:22.349492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23165c-f5b1-42bd-b7a9-a7f8045b5d6d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:22.349808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23165c-f5b1-42bd-b7a9-a7f8045b5d6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:22.350649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23165c-f5b1-42bd-b7a9-a7f8045b5d6d None admin] RBAC: Adding query filter params (role.id=78418cd7aaaf481e94926d572280f81e, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=acc54fd2a9dd44b6b658c44edf170e83) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:22.350953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23165c-f5b1-42bd-b7a9-a7f8045b5d6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:22.352382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23165c-f5b1-42bd-b7a9-a7f8045b5d6d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:22.358810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 633/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:20:22 2022] GET /identity/v3/role_assignments?user.id=acc54fd2a9dd44b6b658c44edf170e83&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=78418cd7aaaf481e94926d572280f81e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:24.516412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7c32-d732-4b60-bf7f-44e857cecee7 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:24.516993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7c32-d732-4b60-bf7f-44e857cecee7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:24.516993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7c32-d732-4b60-bf7f-44e857cecee7 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:24.518357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 614/1247] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:24.523365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f05bac-10d3-4feb-9fb0-a93b0adfe658 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:24.526912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f05bac-10d3-4feb-9fb0-a93b0adfe658 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:24.526912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f05bac-10d3-4feb-9fb0-a93b0adfe658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:24.529193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:24.540877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-97f05bac-10d3-4feb-9fb0-a93b0adfe658 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:24.542037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-97f05bac-10d3-4feb-9fb0-a93b0adfe658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:24.606589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 634/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:20:24.631296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a7c32-d732-4b60-bf7f-44e857cecee7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:24.649750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:24.732082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a7c32-d732-4b60-bf7f-44e857cecee7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:24.766668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5168da-b622-41be-90ee-a345ccdbbd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:24.767373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5168da-b622-41be-90ee-a345ccdbbd5b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:24.767660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5168da-b622-41be-90ee-a345ccdbbd5b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:24.767943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5168da-b622-41be-90ee-a345ccdbbd5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:24.779717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5168da-b622-41be-90ee-a345ccdbbd5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-5e5168da-b622-41be-90ee-a345ccdbbd5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:20:24.787478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:24.789246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:24.790258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 615/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:20:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:24.794874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f05bac-10d3-4feb-9fb0-a93b0adfe658 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:24.899819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f05bac-10d3-4feb-9fb0-a93b0adfe658 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:24.929453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:24.934435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:24.935128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:24.935417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:24.935698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:24.936449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:24.936793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:24.938159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:24.946244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 635/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:20:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:24.952918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5168da-b622-41be-90ee-a345ccdbbd5b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:25.028053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:25.049817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5168da-b622-41be-90ee-a345ccdbbd5b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:25.082055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:25.082832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:25.083119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:25.083399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:25.094196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:25.095624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:25.112637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:25.117231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:25.118366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:25.118366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:25.118366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:25.118366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:25.118366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 616/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:25.119720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:25.221768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c5a92-1daf-4df1-a3a2-545502b9a140 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:25.254279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:25.254937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:25.255224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:25.255512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:25.256226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:25.256550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:25.257920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:25.269917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 636/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:25.275890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:25.373717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc78b70-1813-4a87-ab7c-0f7d3561956b None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:25.398562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:25.406030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:25.409152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:25.409152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:25.409152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:25.414622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:25.416043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:20:25.430025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:25.438455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:25.439666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 617/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:25.440415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:25.490764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:25.533699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66922d59-3cc6-4b80-bbfa-c644ede15abc None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:25.565846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:25.566504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:25.566798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:25.567069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:25.567896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:25.568419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:25.570645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:25.587965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 637/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jun 16 00:20:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:25.593632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:25.695264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e0ebb-fd13-4ccc-9da7-e383a18dec36 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:25.727739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559a829-d864-4876-9cca-e05069a1b312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:25.728371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559a829-d864-4876-9cca-e05069a1b312 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:25.728685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559a829-d864-4876-9cca-e05069a1b312 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:25.728964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559a829-d864-4876-9cca-e05069a1b312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:25.782560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:25.794282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559a829-d864-4876-9cca-e05069a1b312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:25.795714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559a829-d864-4876-9cca-e05069a1b312 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:25.889163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 618/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:20:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:20:25.895123 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:25.990229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ebfb-9988-48e5-aee2-e830fc0df867 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:26.004693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:26.025159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:26.026334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:26.026624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:26.026900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:26.036715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:26.038145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:26.055049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:26.056124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:26.057617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:26.057617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:26.057617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:26.057617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:26.057617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 638/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:26.062519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559a829-d864-4876-9cca-e05069a1b312 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:26.148164 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:26.159278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559a829-d864-4876-9cca-e05069a1b312 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:26.191486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:26.192132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:26.192725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:26.192725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:26.193433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:26.193720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:26.195063 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:26.206237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 619/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:26.211842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:26.312823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b66d6-2a61-48a0-ab81-89268313370f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:26.345712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:26.346488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:26.346771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:26.347046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:26.353705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:26.355137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:20:26.369198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:26.370264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:26.371697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 639/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:20:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:26.376815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:26.472459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c047f-4108-4893-b435-0754b11dd105 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:26.506166 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73100832-d03e-404c-8f29-e58d19d68fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:26.506821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73100832-d03e-404c-8f29-e58d19d68fbb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:26.507104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73100832-d03e-404c-8f29-e58d19d68fbb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:26.507403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73100832-d03e-404c-8f29-e58d19d68fbb None admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:26.508182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73100832-d03e-404c-8f29-e58d19d68fbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:26.508491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73100832-d03e-404c-8f29-e58d19d68fbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:26.509930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73100832-d03e-404c-8f29-e58d19d68fbb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:26.521462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 620/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:20:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:26.527423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:26.553184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:26.625613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d0812-30d3-4462-ab4b-dc5cf0682d84 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:26.660961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a74f3-3c98-49bf-b608-c39482b5a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:26.661625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a74f3-3c98-49bf-b608-c39482b5a65d None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:26.661908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a74f3-3c98-49bf-b608-c39482b5a65d None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:26.662188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a74f3-3c98-49bf-b608-c39482b5a65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:26.663019 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a74f3-3c98-49bf-b608-c39482b5a65d None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=acc54fd2a9dd44b6b658c44edf170e83) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:26.663330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a74f3-3c98-49bf-b608-c39482b5a65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:26.664907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a74f3-3c98-49bf-b608-c39482b5a65d None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:26.670823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 640/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:20:26 2022] GET /identity/v3/role_assignments?user.id=acc54fd2a9dd44b6b658c44edf170e83&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:28.767453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89e647-e7ab-4141-a7c7-9e87e5ea036f None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:28.767981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89e647-e7ab-4141-a7c7-9e87e5ea036f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:28.768103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89e647-e7ab-4141-a7c7-9e87e5ea036f None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:28.769386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 621/1261] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:28.774620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175b40-5c06-408c-9c81-e6e605ea99de None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:28.775051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175b40-5c06-408c-9c81-e6e605ea99de None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:28.775224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175b40-5c06-408c-9c81-e6e605ea99de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:28.796075 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-74175b40-5c06-408c-9c81-e6e605ea99de None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:28.797290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-74175b40-5c06-408c-9c81-e6e605ea99de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:28.862458 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 641/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:20:28.885248 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89e647-e7ab-4141-a7c7-9e87e5ea036f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:28.896217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:28.989486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89e647-e7ab-4141-a7c7-9e87e5ea036f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:29.022363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df02072-2978-4c7f-b972-6ef4e5a2bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:29.023147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df02072-2978-4c7f-b972-6ef4e5a2bf9f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:29.023434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df02072-2978-4c7f-b972-6ef4e5a2bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:29.023716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df02072-2978-4c7f-b972-6ef4e5a2bf9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:29.034197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df02072-2978-4c7f-b972-6ef4e5a2bf9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-6df02072-2978-4c7f-b972-6ef4e5a2bf9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:20:29.041805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:29.045410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:29.046484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 622/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:20:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:29.049821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74175b40-5c06-408c-9c81-e6e605ea99de None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:29.068197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:29.153779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74175b40-5c06-408c-9c81-e6e605ea99de None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:29.187342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:29.188007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:29.188289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:29.188654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:29.189399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:29.189705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:29.191088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:29.199005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 642/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:20:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:29.204765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df02072-2978-4c7f-b972-6ef4e5a2bf9f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:29.268865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:29.302852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df02072-2978-4c7f-b972-6ef4e5a2bf9f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:29.334112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:29.334887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:29.335172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:29.335452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:29.346387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:29.347813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:29.365083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:29.366208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:29.367300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:29.367300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:29.367300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:29.367300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:29.367735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 623/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:29.372261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:29.453360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:29.489447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79f82c-0989-4b8f-990d-b96691fe905f None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:29.523451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:29.524132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:29.524414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:29.524794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:29.525700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:29.526168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:29.528174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:29.539443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 643/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:29.545659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:29.646647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6552085-4927-4570-9161-765539f5bf3c None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:29.665803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:29.680026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:29.680864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:29.681156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:29.681432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] PATH_INFO: `/v3/users/placement` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:29.690355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:29.691778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:20:29.706705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:29.707971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:29.709258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 624/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:29.712956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:29.812429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bceeb-2bab-4203-979e-b2d291cb9ac0 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:29.845717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:29.846374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:29.846673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:29.846928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:29.847785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:29.848092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:29.849537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:29.859091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:29.867092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 644/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jun 16 00:20:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:29.872789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:29.970669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1cad2-823d-47c8-b558-afc2950efbeb None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:30.003083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:30.003855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:30.004140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:30.004417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:30.014749 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:30.016167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:30.033169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:30.034187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:30.035161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:30.035161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:30.035161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:30.035161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:30.035161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 625/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:30.040015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:30.138014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff58f4-a1e8-44d0-b08b-1144119e611a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:30.170758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:30.171455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:30.171737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:30.172010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:30.172902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:30.173212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:30.174564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:30.185697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 645/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:20:30.191477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:30.223661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:30.288632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6afd06-b081-47a5-a74a-8eff580ef697 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:30.322067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:30.322858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:30.323147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:30.323431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:30.330485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:30.331915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:20:30.345834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:30.346909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:30.348116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 626/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:30.352919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:30.409522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:30.448903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536bef-c1db-4195-bf7e-4c7b4a41f9b1 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:30.481985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d5514-b144-46a1-8036-6e0386824dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:30.482570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5514-b144-46a1-8036-6e0386824dcb None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:30.482849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5514-b144-46a1-8036-6e0386824dcb None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:30.483121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5514-b144-46a1-8036-6e0386824dcb None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:30.483896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d5514-b144-46a1-8036-6e0386824dcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:30.484204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d5514-b144-46a1-8036-6e0386824dcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:30.485651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d5514-b144-46a1-8036-6e0386824dcb None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:30.497605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 646/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:20:30.505638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:30.615875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:30.624546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a94a90-6d8a-417e-bd3e-d6527cab5d9d None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:30.657434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de40aef-07f3-4275-9a7a-c2b7601879d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:30.658317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40aef-07f3-4275-9a7a-c2b7601879d3 None admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:30.658598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40aef-07f3-4275-9a7a-c2b7601879d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:30.658870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40aef-07f3-4275-9a7a-c2b7601879d3 None admin] PATH_INFO: `/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/acc54fd2a9dd44b6b658c44edf170e83/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:30.663198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de40aef-07f3-4275-9a7a-c2b7601879d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c36bfe335ce4f4fb3a1ce620eda03af, user_id=acc54fd2a9dd44b6b658c44edf170e83, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:30.663198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de40aef-07f3-4275-9a7a-c2b7601879d3 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:30.889328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 627/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jun 16 00:20:30 2022] PUT /identity/v3/projects/4c36bfe335ce4f4fb3a1ce620eda03af/users/acc54fd2a9dd44b6b658c44edf170e83/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:20:32.990256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cfd6b-507d-41d2-8fcd-e1b9844ee782 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:32.990789 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cfd6b-507d-41d2-8fcd-e1b9844ee782 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:32.990789 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cfd6b-507d-41d2-8fcd-e1b9844ee782 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:32.992091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 647/1274] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:32.997807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d17fab-f4ab-4c0b-bae8-7671ce3ac0ed None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:32.998300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d17fab-f4ab-4c0b-bae8-7671ce3ac0ed None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:32.998396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d17fab-f4ab-4c0b-bae8-7671ce3ac0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:33.015686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e6d17fab-f4ab-4c0b-bae8-7671ce3ac0ed None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:33.017044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e6d17fab-f4ab-4c0b-bae8-7671ce3ac0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:33.084895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 628/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:20:33.107805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cfd6b-507d-41d2-8fcd-e1b9844ee782 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:33.208549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cfd6b-507d-41d2-8fcd-e1b9844ee782 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:33.225240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:33.241624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d883ae-dc89-4086-b0c4-3b553907bda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:33.242348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d883ae-dc89-4086-b0c4-3b553907bda3 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:33.242635 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d883ae-dc89-4086-b0c4-3b553907bda3 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:33.242913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d883ae-dc89-4086-b0c4-3b553907bda3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:33.253316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d883ae-dc89-4086-b0c4-3b553907bda3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-74d883ae-dc89-4086-b0c4-3b553907bda3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:20:33.260884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:33.261987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:33.262899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 648/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jun 16 00:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:33.268190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d17fab-f4ab-4c0b-bae8-7671ce3ac0ed None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:33.349352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:33.367374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d17fab-f4ab-4c0b-bae8-7671ce3ac0ed None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:33.400444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:33.401148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:33.401619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:33.402055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:33.403128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:33.403494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:33.404903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:33.412818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 629/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jun 16 00:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:33.419118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d883ae-dc89-4086-b0c4-3b553907bda3 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:33.519781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d883ae-dc89-4086-b0c4-3b553907bda3 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:33.553304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:33.554084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:33.554366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:33.554640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:33.564767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:33.570308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:33.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:33.583673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:33.584884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:33.584884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:33.584884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:33.584884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:33.584884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 649/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:33.591816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:33.689793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf9a12-835a-42bf-95eb-ad9026c2fe4e None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:33.722651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:33.723301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:33.723587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:33.723864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:33.724617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:33.724925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:33.726364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:33.738700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 630/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:33.744684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:33.774863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:33.841109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccdfd1-2e2e-494b-bf6d-632610440fe6 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:33.873340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:33.874122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:33.874402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:33.874676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:33.881463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:33.882871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:20:33.898643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:33.899820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:33.900843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 650/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:33.906376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:33.910132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:34.004494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a506e03-0d0e-43cd-bade-08f13f04e032 None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:34.038235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:34.038842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:34.039131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:34.039413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:34.040252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:34.040594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:34.042024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:34.059981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 631/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jun 16 00:20:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:34.065869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:34.153680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:34.162559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd52668-2609-49ef-9c54-106c5b77086a None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:34.196138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445efc4-7de3-4ad6-a226-5f0e6d9b3fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:34.196811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445efc4-7de3-4ad6-a226-5f0e6d9b3fc0 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:34.197096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445efc4-7de3-4ad6-a226-5f0e6d9b3fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:34.197373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445efc4-7de3-4ad6-a226-5f0e6d9b3fc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:34.263214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445efc4-7de3-4ad6-a226-5f0e6d9b3fc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:34.264680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445efc4-7de3-4ad6-a226-5f0e6d9b3fc0 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:34.350692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:34.361298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 651/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jun 16 00:20:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:20:34.367187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:34.428216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:34.464940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16db544-3ef2-4c2e-bb76-3c7fa1bc4acb None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:34.501263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:34.502054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:34.502343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:34.502624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:34.512588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:34.514019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:20:34.531621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:34.532762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:34.534184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:20:34.534184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:20:34.534184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:20:34.534184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:34.534184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 632/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:20:34.538590 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445efc4-7de3-4ad6-a226-5f0e6d9b3fc0 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:34.636604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445efc4-7de3-4ad6-a226-5f0e6d9b3fc0 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:34.670226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:34.670890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:34.671171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:34.671448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:34.672159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:34.672466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:34.673855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:34.685291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 652/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jun 16 00:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:34.691005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:34.788670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd7b61-0468-4e00-a5bc-59cb908684bf None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:34.813684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:34.834418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:34.835237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:34.835518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:34.835796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:34.842253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:34.843668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:20:34.857930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:34.861813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: ERROR keystone.server.flask.application Jun 16 00:20:34.862754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 633/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jun 16 00:20:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:34.865158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:34.913616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:34.961851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188f44-b7ce-469c-a51e-00448e5611e1 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:34.999877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b191359-27c6-46fb-9f57-e1aefb42cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:35.000556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b191359-27c6-46fb-9f57-e1aefb42cb25 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:35.000848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b191359-27c6-46fb-9f57-e1aefb42cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:35.001129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b191359-27c6-46fb-9f57-e1aefb42cb25 None admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:35.001914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b191359-27c6-46fb-9f57-e1aefb42cb25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:35.002228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b191359-27c6-46fb-9f57-e1aefb42cb25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:35.003647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b191359-27c6-46fb-9f57-e1aefb42cb25 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:35.016796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 653/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jun 16 00:20:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:20:35.022813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:35.128914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee33bfe-739a-48ba-b002-b13f2fde58ba None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:35.164826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd3955-841a-4b1c-b1f3-5c7d44e763ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:35.165432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd3955-841a-4b1c-b1f3-5c7d44e763ef None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:35.165717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd3955-841a-4b1c-b1f3-5c7d44e763ef None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:35.165994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd3955-841a-4b1c-b1f3-5c7d44e763ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:35.166827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd3955-841a-4b1c-b1f3-5c7d44e763ef None admin] RBAC: Adding query filter params (role.id=11d374b9ca394116bbb541aa3ed7cc2d, scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af, user.id=acc54fd2a9dd44b6b658c44edf170e83) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:35.167138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd3955-841a-4b1c-b1f3-5c7d44e763ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:35.169139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd3955-841a-4b1c-b1f3-5c7d44e763ef None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:35.174987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 634/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jun 16 00:20:35 2022] GET /identity/v3/role_assignments?user.id=acc54fd2a9dd44b6b658c44edf170e83&scope.project.id=4c36bfe335ce4f4fb3a1ce620eda03af&role.id=11d374b9ca394116bbb541aa3ed7cc2d => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:37.346194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f21d8c-998e-41fd-9dbc-78749cf3342c None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:37.346723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f21d8c-998e-41fd-9dbc-78749cf3342c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:37.346833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f21d8c-998e-41fd-9dbc-78749cf3342c None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:37.348173 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 654/1288] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:37.353274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3d2b2-730b-404b-9f21-d788fdbd549e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:37.353651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3d2b2-730b-404b-9f21-d788fdbd549e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:37.353816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3d2b2-730b-404b-9f21-d788fdbd549e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:37.371147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6ad3d2b2-730b-404b-9f21-d788fdbd549e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:37.372343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6ad3d2b2-730b-404b-9f21-d788fdbd549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:37.400731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:37.439883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 635/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:20:37.461813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f21d8c-998e-41fd-9dbc-78749cf3342c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:37.561560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f21d8c-998e-41fd-9dbc-78749cf3342c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:37.571865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:37.595192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:37.596027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:37.596361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:37.596623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:37.597278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:37.598637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:37.610953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:37.612076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:37.613081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 655/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:20:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:37.618146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3d2b2-730b-404b-9f21-d788fdbd549e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:37.715696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3d2b2-730b-404b-9f21-d788fdbd549e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:37.749014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc6002-391c-4d52-8505-b5838dbb9eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:37.749673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc6002-391c-4d52-8505-b5838dbb9eaa None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:37.749961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc6002-391c-4d52-8505-b5838dbb9eaa None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:37.750237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc6002-391c-4d52-8505-b5838dbb9eaa None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:37.750937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc6002-391c-4d52-8505-b5838dbb9eaa None admin] RBAC: Adding query filter params (name=placement) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:37.751240 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc6002-391c-4d52-8505-b5838dbb9eaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:37.752612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc6002-391c-4d52-8505-b5838dbb9eaa None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:37.757473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 636/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Jun 16 00:20:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:37.763925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:37.863339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5f3cb-2159-4dcf-9c80-16a411a74ed0 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:37.895293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa919c-bacf-49c2-b303-980b42f5ba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:37.895989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa919c-bacf-49c2-b303-980b42f5ba2b None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:37.896272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa919c-bacf-49c2-b303-980b42f5ba2b None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:37.896569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa919c-bacf-49c2-b303-980b42f5ba2b None admin] PATH_INFO: `/v3/services` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:37.897270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa919c-bacf-49c2-b303-980b42f5ba2b None admin] RBAC: Adding query filter params (type=placement) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:37.897574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa919c-bacf-49c2-b303-980b42f5ba2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:37.898916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa919c-bacf-49c2-b303-980b42f5ba2b None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:37.903848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 656/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Jun 16 00:20:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:39.982895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c9a20-cfc2-48fa-a07b-e629bbd8030e None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:39.983468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c9a20-cfc2-48fa-a07b-e629bbd8030e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:39.983468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c9a20-cfc2-48fa-a07b-e629bbd8030e None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:39.984927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 637/1293] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:39.990012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0dfb4-b695-48fe-b807-5940dfe5e7ed None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:39.990414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0dfb4-b695-48fe-b807-5940dfe5e7ed None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:39.990575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0dfb4-b695-48fe-b807-5940dfe5e7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:40.007426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e3f0dfb4-b695-48fe-b807-5940dfe5e7ed None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:40.008617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e3f0dfb4-b695-48fe-b807-5940dfe5e7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:40.036942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:40.073308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 657/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:20:40.095809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c9a20-cfc2-48fa-a07b-e629bbd8030e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:40.126952 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:40.194474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c9a20-cfc2-48fa-a07b-e629bbd8030e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:40.226365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e675b1-6d1d-400e-b5ef-28ad6cc534b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:40.227015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e675b1-6d1d-400e-b5ef-28ad6cc534b0 None admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:40.227301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e675b1-6d1d-400e-b5ef-28ad6cc534b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:40.227583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e675b1-6d1d-400e-b5ef-28ad6cc534b0 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:40.228308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e675b1-6d1d-400e-b5ef-28ad6cc534b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:40.229773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e675b1-6d1d-400e-b5ef-28ad6cc534b0 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:40.304830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 638/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jun 16 00:20:40 2022] POST /identity/v3/services => generated 246 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:20:42.433232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9795fe7-09f2-4a33-9de5-2a260958bfc6 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:42.433807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9795fe7-09f2-4a33-9de5-2a260958bfc6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:42.433807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9795fe7-09f2-4a33-9de5-2a260958bfc6 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:42.435385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 658/1296] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:42.441031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc06c38-b3ff-43ac-b458-838d690c68e3 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:42.441475 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc06c38-b3ff-43ac-b458-838d690c68e3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:42.441590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc06c38-b3ff-43ac-b458-838d690c68e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:42.458543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4cc06c38-b3ff-43ac-b458-838d690c68e3 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:42.459705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4cc06c38-b3ff-43ac-b458-838d690c68e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:42.530889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 639/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:20:42.553603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9795fe7-09f2-4a33-9de5-2a260958bfc6 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:42.653192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9795fe7-09f2-4a33-9de5-2a260958bfc6 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:42.686925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:42.687654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:42.687940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:42.688221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:42.688903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:42.690278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:42.702395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:42.703577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:42.704752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 659/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:20:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:20:42.709633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc06c38-b3ff-43ac-b458-838d690c68e3 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:42.727291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:42.804059 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc06c38-b3ff-43ac-b458-838d690c68e3 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:42.837807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:42.838674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:42.838964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:42.839236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:42.839944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] RBAC: Adding query filter params (name=placement) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:42.840275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:42.841656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:42.846791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 640/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Jun 16 00:20:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:20:42.852879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:42.884696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:42.951855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8546aa8-6b71-4a90-a51f-fff67c1efc84 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:42.984121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19416c4d-49b4-40c0-9194-b4738bcd9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:42.984801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19416c4d-49b4-40c0-9194-b4738bcd9a54 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:42.985086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19416c4d-49b4-40c0-9194-b4738bcd9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:42.985362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19416c4d-49b4-40c0-9194-b4738bcd9a54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:42.986140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19416c4d-49b4-40c0-9194-b4738bcd9a54 None admin] RBAC: Adding query filter params (interface=public, service_id=fb8a6a53c89d42f2b72bd163d95cfd31, region_id=RegionOne) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:42.986453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19416c4d-49b4-40c0-9194-b4738bcd9a54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:42.987880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19416c4d-49b4-40c0-9194-b4738bcd9a54 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:42.992743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 660/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jun 16 00:20:42 2022] GET /identity/v3/endpoints?service_id=fb8a6a53c89d42f2b72bd163d95cfd31&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:42.999044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:43.078997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:43.096135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74febd0-0295-46d9-aed8-f574e46d420f None admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:43.130556 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fccee-4db7-4fd6-ada9-0390163342cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:43.131249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fccee-4db7-4fd6-ada9-0390163342cb None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:43.131536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fccee-4db7-4fd6-ada9-0390163342cb None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:43.131818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fccee-4db7-4fd6-ada9-0390163342cb None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:43.132502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fccee-4db7-4fd6-ada9-0390163342cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:43.133980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fccee-4db7-4fd6-ada9-0390163342cb None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:43.140662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 641/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jun 16 00:20:42 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:20:45.254314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319ebb-649b-4415-9f67-9bda3524bdf1 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:45.254871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319ebb-649b-4415-9f67-9bda3524bdf1 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:45.254871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319ebb-649b-4415-9f67-9bda3524bdf1 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:45.256219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 661/1302] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:45.261327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75906256-6292-4b1a-8a06-aa1c766f086c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:45.261745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75906256-6292-4b1a-8a06-aa1c766f086c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:45.261895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75906256-6292-4b1a-8a06-aa1c766f086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:45.279396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-75906256-6292-4b1a-8a06-aa1c766f086c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:45.280610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-75906256-6292-4b1a-8a06-aa1c766f086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:45.345399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 642/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:20:45.368023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d319ebb-649b-4415-9f67-9bda3524bdf1 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:45.461402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:45.468465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d319ebb-649b-4415-9f67-9bda3524bdf1 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:45.501319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:45.502111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:45.502393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:45.502669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:45.503315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:45.504754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:20:45.517683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:20:45.518914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application with Lock( Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: ERROR keystone.server.flask.application Jun 16 00:20:45.520048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 662/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jun 16 00:20:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:20:45.524860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75906256-6292-4b1a-8a06-aa1c766f086c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:45.602734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:45.625147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75906256-6292-4b1a-8a06-aa1c766f086c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:45.658221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95698033-c78d-4793-9531-d8e0414f3330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:45.658878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698033-c78d-4793-9531-d8e0414f3330 None admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:45.659161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698033-c78d-4793-9531-d8e0414f3330 None admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:45.659436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698033-c78d-4793-9531-d8e0414f3330 None admin] PATH_INFO: `/v3/services` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:45.660136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698033-c78d-4793-9531-d8e0414f3330 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:20:45.660446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698033-c78d-4793-9531-d8e0414f3330 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:45.661816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698033-c78d-4793-9531-d8e0414f3330 None admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:45.666780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 643/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Jun 16 00:20:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:20:45.673137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:45.768325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a357-8576-4308-b522-4625f2d8c2d7 None admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:45.802006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608897ae-05ff-4c16-91b3-85eb70faa9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:45.802645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608897ae-05ff-4c16-91b3-85eb70faa9a0 None admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:45.802928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608897ae-05ff-4c16-91b3-85eb70faa9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:45.803202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608897ae-05ff-4c16-91b3-85eb70faa9a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:45.803939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608897ae-05ff-4c16-91b3-85eb70faa9a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:45.805307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608897ae-05ff-4c16-91b3-85eb70faa9a0 None admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:45.883478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 663/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jun 16 00:20:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:20:55.663769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=30 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:55.877633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:27.259403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b8d1-8003-41b1-bc3b-bee43ba3b0ab None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:27.260028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b8d1-8003-41b1-bc3b-bee43ba3b0ab None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:27.260028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b8d1-8003-41b1-bc3b-bee43ba3b0ab None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:27.261401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 644/1307] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:27.266921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804ff6-f558-4300-a6e7-665c32f135d1 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:27.267777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804ff6-f558-4300-a6e7-665c32f135d1 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:27.268280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804ff6-f558-4300-a6e7-665c32f135d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:27.382055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-46804ff6-f558-4300-a6e7-665c32f135d1 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:27.382055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-46804ff6-f558-4300-a6e7-665c32f135d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:27.450753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 664/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:27.504075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36860d-08b0-48ad-a869-7074ed6bcad9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:27.504908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36860d-08b0-48ad-a869-7074ed6bcad9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:27.505571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36860d-08b0-48ad-a869-7074ed6bcad9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:27.510026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 645/1309] 10.222.0.20 () {54 vars in 964 bytes} [Thu Jun 16 00:26:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:27.518997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238e558-4de6-4e25-ae13-c18efee429f0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:27.519552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238e558-4de6-4e25-ae13-c18efee429f0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:27.520126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238e558-4de6-4e25-ae13-c18efee429f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:27.631267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7238e558-4de6-4e25-ae13-c18efee429f0 None None] MFA Rules not processed for user `96df1ee2cbc14bcc8a8a250fd8f349dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:27.631267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7238e558-4de6-4e25-ae13-c18efee429f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:27.661154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:27.742827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 665/1310] 10.222.0.20 () {58 vars in 1061 bytes} [Thu Jun 16 00:26:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:27.753149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36860d-08b0-48ad-a869-7074ed6bcad9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:27.754950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5b36860d-08b0-48ad-a869-7074ed6bcad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:27.890350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36860d-08b0-48ad-a869-7074ed6bcad9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:27.969856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085c2c4-9f07-44a1-b01a-e94c4405c3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:27.970539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c2c4-9f07-44a1-b01a-e94c4405c3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:27.970825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c2c4-9f07-44a1-b01a-e94c4405c3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:27.971104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c2c4-9f07-44a1-b01a-e94c4405c3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:27.973545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8085c2c4-9f07-44a1-b01a-e94c4405c3a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:28.031345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:28.087237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085c2c4-9f07-44a1-b01a-e94c4405c3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:28.088722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085c2c4-9f07-44a1-b01a-e94c4405c3a3 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:28.107612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 646/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:30.529610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f00cce-fc18-412b-8bc4-bdb9d03c4a9c None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:30.530145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f00cce-fc18-412b-8bc4-bdb9d03c4a9c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:30.530272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f00cce-fc18-412b-8bc4-bdb9d03c4a9c None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:30.531542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 666/1312] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:30.536721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6baf2c-ade2-41cb-a78c-75aba6d5ed87 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:30.537717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6baf2c-ade2-41cb-a78c-75aba6d5ed87 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:30.537891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6baf2c-ade2-41cb-a78c-75aba6d5ed87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:30.618649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9f6baf2c-ade2-41cb-a78c-75aba6d5ed87 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:30.619847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9f6baf2c-ade2-41cb-a78c-75aba6d5ed87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:30.720683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 647/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:30.754503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f00cce-fc18-412b-8bc4-bdb9d03c4a9c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:30.756693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b8f00cce-fc18-412b-8bc4-bdb9d03c4a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:30.888434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f00cce-fc18-412b-8bc4-bdb9d03c4a9c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:30.974691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:30.977983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed36b6-0a23-4980-a0a0-fd1a9f200aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:30.978636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed36b6-0a23-4980-a0a0-fd1a9f200aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:30.978915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed36b6-0a23-4980-a0a0-fd1a9f200aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:30.979185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed36b6-0a23-4980-a0a0-fd1a9f200aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:30.980883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-45ed36b6-0a23-4980-a0a0-fd1a9f200aa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:31.099211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed36b6-0a23-4980-a0a0-fd1a9f200aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:31.100696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed36b6-0a23-4980-a0a0-fd1a9f200aa4 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:31.124055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 667/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:33.891321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1db13-e308-4a82-9ab1-62512807d51e None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:33.891838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1db13-e308-4a82-9ab1-62512807d51e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:33.891838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1db13-e308-4a82-9ab1-62512807d51e None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:33.893249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 648/1315] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:33.898984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6bf14-4f45-4732-99ae-ba88a395e969 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:33.899409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6bf14-4f45-4732-99ae-ba88a395e969 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:33.899539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6bf14-4f45-4732-99ae-ba88a395e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:33.962505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-04c6bf14-4f45-4732-99ae-ba88a395e969 None None] MFA Rules not processed for user `714d13e3ae3b43d2a5b81b6f4fc5d980`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:33.963699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-04c6bf14-4f45-4732-99ae-ba88a395e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:34.014730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:34.059362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 668/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:33 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:34.098323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1db13-e308-4a82-9ab1-62512807d51e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:34.099904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-65b1db13-e308-4a82-9ab1-62512807d51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:34.109529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:34.207174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1db13-e308-4a82-9ab1-62512807d51e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:34.277427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a0927-90b4-4498-8bba-de86e2f57994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:34.278066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a0927-90b4-4498-8bba-de86e2f57994 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:34.278345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a0927-90b4-4498-8bba-de86e2f57994 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:34.278616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a0927-90b4-4498-8bba-de86e2f57994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:34.280250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-027a0927-90b4-4498-8bba-de86e2f57994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:34.340984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:34.386476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a0927-90b4-4498-8bba-de86e2f57994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:34.387887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a0927-90b4-4498-8bba-de86e2f57994 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:34.405202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 649/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:37.288398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d03621-07e2-43f1-b531-e594c196cecf None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:37.288928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d03621-07e2-43f1-b531-e594c196cecf None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:37.289024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d03621-07e2-43f1-b531-e594c196cecf None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:37.290364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 669/1318] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:37.295622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34791-aabc-49f8-975a-110bda6f95e7 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:37.296020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34791-aabc-49f8-975a-110bda6f95e7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:37.296192 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34791-aabc-49f8-975a-110bda6f95e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:37.359630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-71e34791-aabc-49f8-975a-110bda6f95e7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:37.360872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-71e34791-aabc-49f8-975a-110bda6f95e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:37.454970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 650/1319] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:37.493418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d03621-07e2-43f1-b531-e594c196cecf None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:37.494936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c1d03621-07e2-43f1-b531-e594c196cecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:37.597597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d03621-07e2-43f1-b531-e594c196cecf None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:37.669665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ba19e-14dc-4e91-a7f2-9a8f831fde1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:37.670305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ba19e-14dc-4e91-a7f2-9a8f831fde1a service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:37.670586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ba19e-14dc-4e91-a7f2-9a8f831fde1a service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:37.670859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ba19e-14dc-4e91-a7f2-9a8f831fde1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:37.672490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1b7ba19e-14dc-4e91-a7f2-9a8f831fde1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:37.680479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:37.793266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ba19e-14dc-4e91-a7f2-9a8f831fde1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:37.794693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ba19e-14dc-4e91-a7f2-9a8f831fde1a service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:37.812495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 670/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:40.904253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650cc2-c52d-4bd3-a43d-290406813d70 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:40.904789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650cc2-c52d-4bd3-a43d-290406813d70 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:40.904789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650cc2-c52d-4bd3-a43d-290406813d70 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:40.906145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 651/1321] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:40.911354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947337a6-e15d-403a-9c52-50d2060eb73a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:40.911771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947337a6-e15d-403a-9c52-50d2060eb73a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:40.911918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947337a6-e15d-403a-9c52-50d2060eb73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:40.980640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-947337a6-e15d-403a-9c52-50d2060eb73a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:40.981884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-947337a6-e15d-403a-9c52-50d2060eb73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:41.044815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:41.086780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 671/1322] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:41.122850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650cc2-c52d-4bd3-a43d-290406813d70 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:41.124407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6d650cc2-c52d-4bd3-a43d-290406813d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:41.143806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:41.230745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650cc2-c52d-4bd3-a43d-290406813d70 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:41.303647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1106b-64cd-4b5b-a804-97a48e180bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:41.304302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1106b-64cd-4b5b-a804-97a48e180bbd service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:41.304620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1106b-64cd-4b5b-a804-97a48e180bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:41.304901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1106b-64cd-4b5b-a804-97a48e180bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:41.306555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-61c1106b-64cd-4b5b-a804-97a48e180bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:41.377056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:41.415066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1106b-64cd-4b5b-a804-97a48e180bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:41.416507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1106b-64cd-4b5b-a804-97a48e180bbd service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:41.436646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 652/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:43.653359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eb0e3-d9e2-4d05-a8ee-6773f75a0895 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:43.653910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eb0e3-d9e2-4d05-a8ee-6773f75a0895 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:43.653910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eb0e3-d9e2-4d05-a8ee-6773f75a0895 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:43.655278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 672/1324] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:43.660548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece380e0-b627-4b9d-b8ed-b48b2022c872 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:43.660959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece380e0-b627-4b9d-b8ed-b48b2022c872 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:43.661118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece380e0-b627-4b9d-b8ed-b48b2022c872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:43.727606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ece380e0-b627-4b9d-b8ed-b48b2022c872 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:43.728836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ece380e0-b627-4b9d-b8ed-b48b2022c872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:43.829727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 653/1325] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:43.864869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eb0e3-d9e2-4d05-a8ee-6773f75a0895 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:43.867342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3f3eb0e3-d9e2-4d05-a8ee-6773f75a0895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:43.984826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eb0e3-d9e2-4d05-a8ee-6773f75a0895 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:44.059397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec79b0-f59b-4efb-9b47-8301e63cfe2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:44.060055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec79b0-f59b-4efb-9b47-8301e63cfe2a service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:44.060341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec79b0-f59b-4efb-9b47-8301e63cfe2a service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:44.060667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec79b0-f59b-4efb-9b47-8301e63cfe2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:44.062320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-40ec79b0-f59b-4efb-9b47-8301e63cfe2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:44.080097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:44.179337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec79b0-f59b-4efb-9b47-8301e63cfe2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:44.180835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec79b0-f59b-4efb-9b47-8301e63cfe2a service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:44.198829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 673/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:46.389315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d54d9f-9e0d-4d6b-b83d-10d6e1d67a0d None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:46.389840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d54d9f-9e0d-4d6b-b83d-10d6e1d67a0d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:46.389957 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d54d9f-9e0d-4d6b-b83d-10d6e1d67a0d None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:46.391274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 654/1327] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:46.396634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1c517-a34e-4d85-8980-39cab3f140bd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:46.397097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1c517-a34e-4d85-8980-39cab3f140bd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:46.397247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1c517-a34e-4d85-8980-39cab3f140bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:46.461751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-dfc1c517-a34e-4d85-8980-39cab3f140bd None None] MFA Rules not processed for user `714d13e3ae3b43d2a5b81b6f4fc5d980`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:46.462950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dfc1c517-a34e-4d85-8980-39cab3f140bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:46.544676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:46.584435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 674/1328] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:46.622032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d54d9f-9e0d-4d6b-b83d-10d6e1d67a0d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:46.623615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-01d54d9f-9e0d-4d6b-b83d-10d6e1d67a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:46.671837 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:46.749427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d54d9f-9e0d-4d6b-b83d-10d6e1d67a0d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:46.824247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0d64-ae30-46cb-b3a0-20cc2e3eeb81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:46.824938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0d64-ae30-46cb-b3a0-20cc2e3eeb81 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:46.825227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0d64-ae30-46cb-b3a0-20cc2e3eeb81 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:46.825509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0d64-ae30-46cb-b3a0-20cc2e3eeb81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:46.827149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ddeb0d64-ae30-46cb-b3a0-20cc2e3eeb81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:46.913653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:46.944854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb0d64-ae30-46cb-b3a0-20cc2e3eeb81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:46.946302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb0d64-ae30-46cb-b3a0-20cc2e3eeb81 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:46.965423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 655/1329] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:49.516247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b373-181b-4c68-8ccd-e532f7116b48 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:49.516798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b373-181b-4c68-8ccd-e532f7116b48 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:49.517254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b373-181b-4c68-8ccd-e532f7116b48 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:49.518301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 675/1330] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:49.523555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e872ebb-8856-47d5-90fd-07e7c6d13aa5 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:49.523975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e872ebb-8856-47d5-90fd-07e7c6d13aa5 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:49.524114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e872ebb-8856-47d5-90fd-07e7c6d13aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:49.590549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-2e872ebb-8856-47d5-90fd-07e7c6d13aa5 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:49.591722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2e872ebb-8856-47d5-90fd-07e7c6d13aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:49.688882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 656/1331] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:49.723327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b373-181b-4c68-8ccd-e532f7116b48 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:49.725417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4988b373-181b-4c68-8ccd-e532f7116b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:49.830719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b373-181b-4c68-8ccd-e532f7116b48 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:49.902453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c0b7f-2c04-49bc-8675-cb799ee49722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:49.903018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c0b7f-2c04-49bc-8675-cb799ee49722 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:49.903307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c0b7f-2c04-49bc-8675-cb799ee49722 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:49.903589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c0b7f-2c04-49bc-8675-cb799ee49722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:49.905278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1e3c0b7f-2c04-49bc-8675-cb799ee49722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:49.935717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:50.015715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c0b7f-2c04-49bc-8675-cb799ee49722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:50.017278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c0b7f-2c04-49bc-8675-cb799ee49722 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:50.035359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 676/1332] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:52.202901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b3fd-a3ea-4621-933f-5493dc1fd89b None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:52.203468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b3fd-a3ea-4621-933f-5493dc1fd89b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:52.203468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b3fd-a3ea-4621-933f-5493dc1fd89b None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:52.204871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 657/1333] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:52.210584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f3a-e776-4434-82e3-7a225fe0f9ab None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:52.210961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f3a-e776-4434-82e3-7a225fe0f9ab None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:52.211153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f3a-e776-4434-82e3-7a225fe0f9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:52.274837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-19198f3a-e776-4434-82e3-7a225fe0f9ab None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:52.276034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-19198f3a-e776-4434-82e3-7a225fe0f9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:52.348685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:52.370594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 677/1334] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:52.410669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5b3fd-a3ea-4621-933f-5493dc1fd89b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:52.412232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-74e5b3fd-a3ea-4621-933f-5493dc1fd89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:52.451762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:52.513686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5b3fd-a3ea-4621-933f-5493dc1fd89b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:52.593250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b346-b59e-4303-85a7-cfb51077e094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:52.593929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b346-b59e-4303-85a7-cfb51077e094 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:52.594210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b346-b59e-4303-85a7-cfb51077e094 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:52.594483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b346-b59e-4303-85a7-cfb51077e094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:52.596208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4436b346-b59e-4303-85a7-cfb51077e094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:52.677587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:52.703303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436b346-b59e-4303-85a7-cfb51077e094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:52.704762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436b346-b59e-4303-85a7-cfb51077e094 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:52.723154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 658/1335] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:26:55.413094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f59b4-a452-465b-9073-9c1b2eb70ccd None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:55.413662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f59b4-a452-465b-9073-9c1b2eb70ccd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:55.413662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f59b4-a452-465b-9073-9c1b2eb70ccd None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:55.415121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 678/1336] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:26:55.420207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2493df-9c14-4f41-a4b4-0c704487aee2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:55.420605 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2493df-9c14-4f41-a4b4-0c704487aee2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:55.420816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2493df-9c14-4f41-a4b4-0c704487aee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:55.483969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ce2493df-9c14-4f41-a4b4-0c704487aee2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:55.485198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ce2493df-9c14-4f41-a4b4-0c704487aee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:55.581527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 659/1337] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:26:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:55.617285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f59b4-a452-465b-9073-9c1b2eb70ccd None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:55.619690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6c7f59b4-a452-465b-9073-9c1b2eb70ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:55.725396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f59b4-a452-465b-9073-9c1b2eb70ccd None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:55.800114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af33eaa-09cf-44bc-9ce3-10345a3b014e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:55.800789 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af33eaa-09cf-44bc-9ce3-10345a3b014e service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:55.801076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af33eaa-09cf-44bc-9ce3-10345a3b014e service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:55.801358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af33eaa-09cf-44bc-9ce3-10345a3b014e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:55.803499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5af33eaa-09cf-44bc-9ce3-10345a3b014e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:55.838948 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:55.910651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af33eaa-09cf-44bc-9ce3-10345a3b014e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:55.912070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af33eaa-09cf-44bc-9ce3-10345a3b014e service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:55.930472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 679/1338] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:26:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:00.293123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8671c-afa3-441d-85f6-259fa14f1f48 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.293780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8671c-afa3-441d-85f6-259fa14f1f48 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.293780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8671c-afa3-441d-85f6-259fa14f1f48 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.295086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 660/1339] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:00.300371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713add7-f379-4949-81e4-7d1975c9e676 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.300834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713add7-f379-4949-81e4-7d1975c9e676 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.300970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713add7-f379-4949-81e4-7d1975c9e676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.383366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b713add7-f379-4949-81e4-7d1975c9e676 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.384684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b713add7-f379-4949-81e4-7d1975c9e676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.480448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:00.494242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 680/1340] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.532473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8671c-afa3-441d-85f6-259fa14f1f48 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.534925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a4f8671c-afa3-441d-85f6-259fa14f1f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.588157 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:00.637638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8671c-afa3-441d-85f6-259fa14f1f48 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.641420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db480b84-0071-4913-9e80-189de0dda25b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.642067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480b84-0071-4913-9e80-189de0dda25b service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.642353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480b84-0071-4913-9e80-189de0dda25b service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.642635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db480b84-0071-4913-9e80-189de0dda25b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.652872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db480b84-0071-4913-9e80-189de0dda25b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.654297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db480b84-0071-4913-9e80-189de0dda25b service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.663902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 661/1341] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:03.800315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb3f46-5a25-4ffd-b970-1aa5acb3ddec None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.800881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb3f46-5a25-4ffd-b970-1aa5acb3ddec None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.800881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb3f46-5a25-4ffd-b970-1aa5acb3ddec None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.802269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 681/1342] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:03.807748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e7fa-d807-4044-b930-5ef98788d074 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.808196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e7fa-d807-4044-b930-5ef98788d074 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.808307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e7fa-d807-4044-b930-5ef98788d074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.836885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-1c17e7fa-d807-4044-b930-5ef98788d074 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:03.838470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1c17e7fa-d807-4044-b930-5ef98788d074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:03.916508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 662/1343] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:03.954205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb3f46-5a25-4ffd-b970-1aa5acb3ddec None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:03.970801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb3f46-5a25-4ffd-b970-1aa5acb3ddec None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:03.974530 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041d355-4191-48e7-80ff-38dbbc6c2a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:03.975210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d355-4191-48e7-80ff-38dbbc6c2a36 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.975492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d355-4191-48e7-80ff-38dbbc6c2a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.975769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d355-4191-48e7-80ff-38dbbc6c2a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.985771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041d355-4191-48e7-80ff-38dbbc6c2a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:03.987224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041d355-4191-48e7-80ff-38dbbc6c2a36 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:03.997677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 682/1344] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:08.330438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501b5b-8184-4365-9859-fdec3f99fa67 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:08.330914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501b5b-8184-4365-9859-fdec3f99fa67 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:08.331036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501b5b-8184-4365-9859-fdec3f99fa67 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:08.332403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 663/1345] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:08.338214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011572b1-0c75-4ec2-bf4d-ff6bf197ce13 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:08.338619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011572b1-0c75-4ec2-bf4d-ff6bf197ce13 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:08.338778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011572b1-0c75-4ec2-bf4d-ff6bf197ce13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:08.361496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-011572b1-0c75-4ec2-bf4d-ff6bf197ce13 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:08.362827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-011572b1-0c75-4ec2-bf4d-ff6bf197ce13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:08.371119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 683/1346] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:08.409183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46501b5b-8184-4365-9859-fdec3f99fa67 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:08.422464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46501b5b-8184-4365-9859-fdec3f99fa67 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:08.426082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd0a74-fa10-4c37-b3f2-a2a95b23c907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:08.426811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0a74-fa10-4c37-b3f2-a2a95b23c907 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:08.427165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0a74-fa10-4c37-b3f2-a2a95b23c907 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:08.427515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0a74-fa10-4c37-b3f2-a2a95b23c907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:08.437874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd0a74-fa10-4c37-b3f2-a2a95b23c907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:08.439437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd0a74-fa10-4c37-b3f2-a2a95b23c907 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:08.450750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 664/1347] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:13.475566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038b438-11bc-4efd-98df-373c050f822f None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.476576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038b438-11bc-4efd-98df-373c050f822f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.477167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038b438-11bc-4efd-98df-373c050f822f None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.478783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 684/1348] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:13.483974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cbd0f-1774-4a8b-be8d-d88fc0a2c50b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.485048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cbd0f-1774-4a8b-be8d-d88fc0a2c50b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.485559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cbd0f-1774-4a8b-be8d-d88fc0a2c50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.560131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a18cbd0f-1774-4a8b-be8d-d88fc0a2c50b None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:13.561624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a18cbd0f-1774-4a8b-be8d-d88fc0a2c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:13.589729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:13.639487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 665/1349] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:27:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:17.793934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aada9a9-1756-48a9-9d8d-260e753ae344 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.794463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aada9a9-1756-48a9-9d8d-260e753ae344 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.794463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aada9a9-1756-48a9-9d8d-260e753ae344 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.796007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 685/1350] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:17.800833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2499be5-c61e-476b-9292-b9ae494ffba6 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.801271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2499be5-c61e-476b-9292-b9ae494ffba6 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.801409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2499be5-c61e-476b-9292-b9ae494ffba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.820135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-b2499be5-c61e-476b-9292-b9ae494ffba6 None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:17.821366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b2499be5-c61e-476b-9292-b9ae494ffba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.829998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 666/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:27:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:18.353220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=13 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:19.463281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f7908-a9d3-411d-a3a5-1231d09e7f04 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.463932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f7908-a9d3-411d-a3a5-1231d09e7f04 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.463932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f7908-a9d3-411d-a3a5-1231d09e7f04 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.465239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 686/1352] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:19.470953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bb0df-c6ef-4ead-888e-96c88d4da629 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.471393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bb0df-c6ef-4ead-888e-96c88d4da629 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.471647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bb0df-c6ef-4ead-888e-96c88d4da629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.490682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-516bb0df-c6ef-4ead-888e-96c88d4da629 None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:19.491955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-516bb0df-c6ef-4ead-888e-96c88d4da629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:19.500421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 667/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:27:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:23.320767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35bd8-2ed5-469e-88af-109e2deff142 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.322244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35bd8-2ed5-469e-88af-109e2deff142 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.322525 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35bd8-2ed5-469e-88af-109e2deff142 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.324034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 687/1354] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:23.329776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9c24e-1b0f-4f16-b145-2632d33658fd None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.330222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9c24e-1b0f-4f16-b145-2632d33658fd None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.330346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9c24e-1b0f-4f16-b145-2632d33658fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.348126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-18d9c24e-1b0f-4f16-b145-2632d33658fd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:23.349909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-18d9c24e-1b0f-4f16-b145-2632d33658fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.358501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 668/1355] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:23.420799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124b16-0a10-47a3-8f85-0b37717fb7d5 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.421311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124b16-0a10-47a3-8f85-0b37717fb7d5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.421732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124b16-0a10-47a3-8f85-0b37717fb7d5 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.423566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 688/1356] 10.222.0.20 () {54 vars in 961 bytes} [Thu Jun 16 00:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:23.431785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4551a-d377-4a4c-a045-237b5ba326e1 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.432274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4551a-d377-4a4c-a045-237b5ba326e1 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.432981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4551a-d377-4a4c-a045-237b5ba326e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.486764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d6f4551a-d377-4a4c-a045-237b5ba326e1 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:23.488058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d6f4551a-d377-4a4c-a045-237b5ba326e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.587924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 669/1357] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Jun 16 00:27:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:23.596290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124b16-0a10-47a3-8f85-0b37717fb7d5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:23.612916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124b16-0a10-47a3-8f85-0b37717fb7d5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:23.618887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6ff56-cc5d-404b-9ff9-8c07bcfc9e7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:23.621099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6ff56-cc5d-404b-9ff9-8c07bcfc9e7d service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.621419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6ff56-cc5d-404b-9ff9-8c07bcfc9e7d service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.621694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6ff56-cc5d-404b-9ff9-8c07bcfc9e7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.631005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6ff56-cc5d-404b-9ff9-8c07bcfc9e7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:23.633000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6ff56-cc5d-404b-9ff9-8c07bcfc9e7d service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:23.644570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 689/1358] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:25.972202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662508b2-fe05-486b-97c5-d2abd64f101c None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.972760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662508b2-fe05-486b-97c5-d2abd64f101c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.972760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662508b2-fe05-486b-97c5-d2abd64f101c None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.974265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 670/1359] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:25.979236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b139-cf10-413e-a832-16973626e0bd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.979662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b139-cf10-413e-a832-16973626e0bd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.979812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b139-cf10-413e-a832-16973626e0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.998195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f976b139-cf10-413e-a832-16973626e0bd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:25.999401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f976b139-cf10-413e-a832-16973626e0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:26.007870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 690/1360] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:26.058386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662508b2-fe05-486b-97c5-d2abd64f101c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:26.076068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662508b2-fe05-486b-97c5-d2abd64f101c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:26.082217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb24bc-849a-4c29-b7c1-fdc5f9974afa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:26.083358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb24bc-849a-4c29-b7c1-fdc5f9974afa service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:26.088006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb24bc-849a-4c29-b7c1-fdc5f9974afa service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:26.088006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb24bc-849a-4c29-b7c1-fdc5f9974afa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:26.093838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb24bc-849a-4c29-b7c1-fdc5f9974afa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:26.095504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb24bc-849a-4c29-b7c1-fdc5f9974afa service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:26.107485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 671/1361] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:28.717237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8af1-b926-4051-8f93-16493a06bcff None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.717709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8af1-b926-4051-8f93-16493a06bcff None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.717836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8af1-b926-4051-8f93-16493a06bcff None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.719217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 691/1362] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:28.724490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57b0ad-484b-46d9-80c1-9793b8f8849b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.724959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57b0ad-484b-46d9-80c1-9793b8f8849b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.725212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57b0ad-484b-46d9-80c1-9793b8f8849b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.743625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-3c57b0ad-484b-46d9-80c1-9793b8f8849b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:28.744854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3c57b0ad-484b-46d9-80c1-9793b8f8849b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:28.752932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 672/1363] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:28.798929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa8558-87e4-4800-a2fa-572c063d9c8b None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.799223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa8558-87e4-4800-a2fa-572c063d9c8b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.799553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa8558-87e4-4800-a2fa-572c063d9c8b None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.800978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 692/1364] 10.222.0.20 () {54 vars in 961 bytes} [Thu Jun 16 00:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:28.806975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6b1d-57c9-4989-bc3a-638668fbb83c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.807265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6b1d-57c9-4989-bc3a-638668fbb83c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.807825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b6b1d-57c9-4989-bc3a-638668fbb83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.828776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-0e8b6b1d-57c9-4989-bc3a-638668fbb83c None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:28.829950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-0e8b6b1d-57c9-4989-bc3a-638668fbb83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:28.840360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 673/1365] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Jun 16 00:27:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:28.848224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa8558-87e4-4800-a2fa-572c063d9c8b None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:28.862746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa8558-87e4-4800-a2fa-572c063d9c8b None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:28.868691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970560a-e32a-4ef7-8862-509fe83172d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:28.869660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970560a-e32a-4ef7-8862-509fe83172d0 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.870298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970560a-e32a-4ef7-8862-509fe83172d0 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.870298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970560a-e32a-4ef7-8862-509fe83172d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.879048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970560a-e32a-4ef7-8862-509fe83172d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:28.881248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970560a-e32a-4ef7-8862-509fe83172d0 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:28.890695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 693/1366] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:31.581286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c525b-d705-447b-8056-292fb4071b98 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.584220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c525b-d705-447b-8056-292fb4071b98 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.584531 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c525b-d705-447b-8056-292fb4071b98 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.586084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 674/1367] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:31.594825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a039b19-5651-46e0-8c90-b66c3c0d7d7e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.595890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d87342-6cb5-4104-ab53-652e9881e67a None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.596360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d87342-6cb5-4104-ab53-652e9881e67a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.596653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a039b19-5651-46e0-8c90-b66c3c0d7d7e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.596763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d87342-6cb5-4104-ab53-652e9881e67a None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.596924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a039b19-5651-46e0-8c90-b66c3c0d7d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.597997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 675/1368] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:27:31.603303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d834d-a83c-4eba-bd5c-2ab5a0720ef5 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.603588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d834d-a83c-4eba-bd5c-2ab5a0720ef5 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.603869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d834d-a83c-4eba-bd5c-2ab5a0720ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.619965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-3a039b19-5651-46e0-8c90-b66c3c0d7d7e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:31.621544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3a039b19-5651-46e0-8c90-b66c3c0d7d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:31.621775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e63d834d-a83c-4eba-bd5c-2ab5a0720ef5 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:31.622929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e63d834d-a83c-4eba-bd5c-2ab5a0720ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:31.630086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 694/1369] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:31.633924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 676/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:31.674142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a039b19-5651-46e0-8c90-b66c3c0d7d7e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:31.688441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a039b19-5651-46e0-8c90-b66c3c0d7d7e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:31.694191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979b63e-1bd3-4e24-99bc-9028be72fd41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:31.694957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b63e-1bd3-4e24-99bc-9028be72fd41 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.695351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b63e-1bd3-4e24-99bc-9028be72fd41 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.695736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b63e-1bd3-4e24-99bc-9028be72fd41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.705144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979b63e-1bd3-4e24-99bc-9028be72fd41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:31.706838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979b63e-1bd3-4e24-99bc-9028be72fd41 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:31.717700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 695/1371] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:31.932916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552e987-d660-405a-a9c4-fbc4790cec27 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.933212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552e987-d660-405a-a9c4-fbc4790cec27 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.933449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552e987-d660-405a-a9c4-fbc4790cec27 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.934796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 677/1372] 10.222.0.20 () {54 vars in 963 bytes} [Thu Jun 16 00:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:31.940568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee684-499b-4745-b52e-15e5dc540736 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.940568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee684-499b-4745-b52e-15e5dc540736 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.940906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee684-499b-4745-b52e-15e5dc540736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.005693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-eefee684-499b-4745-b52e-15e5dc540736 None None] MFA Rules not processed for user `fff82e4b1a4248fcbb3566bb7bc84d26`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:32.007500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-eefee684-499b-4745-b52e-15e5dc540736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:32.081595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 696/1373] 10.222.0.20 () {58 vars in 1060 bytes} [Thu Jun 16 00:27:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:32.090112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552e987-d660-405a-a9c4-fbc4790cec27 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:32.103869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552e987-d660-405a-a9c4-fbc4790cec27 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:32.110651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf06674-ed62-49c4-97f1-6b015524fb73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:32.110651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf06674-ed62-49c4-97f1-6b015524fb73 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.110651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf06674-ed62-49c4-97f1-6b015524fb73 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.110651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf06674-ed62-49c4-97f1-6b015524fb73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.119515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf06674-ed62-49c4-97f1-6b015524fb73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:32.121295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf06674-ed62-49c4-97f1-6b015524fb73 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:32.131969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 678/1374] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:34.075665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656037ed-d921-41fd-961e-4f4de2b8d740 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.076219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656037ed-d921-41fd-961e-4f4de2b8d740 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.076219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656037ed-d921-41fd-961e-4f4de2b8d740 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.077639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 697/1375] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:27:34.082943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d821131-607d-48aa-8782-7ecbe56b134b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.083405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d821131-607d-48aa-8782-7ecbe56b134b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.083531 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d821131-607d-48aa-8782-7ecbe56b134b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.094841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:34.106089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-5d821131-607d-48aa-8782-7ecbe56b134b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:34.107287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5d821131-607d-48aa-8782-7ecbe56b134b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.116385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 679/1376] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:34.160169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656037ed-d921-41fd-961e-4f4de2b8d740 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:34.175338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656037ed-d921-41fd-961e-4f4de2b8d740 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:34.182068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a17881-f4a6-4ad2-929f-caacb6346901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:34.182978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a17881-f4a6-4ad2-929f-caacb6346901 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.183267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a17881-f4a6-4ad2-929f-caacb6346901 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.183552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a17881-f4a6-4ad2-929f-caacb6346901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.192147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a17881-f4a6-4ad2-929f-caacb6346901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:34.194341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a17881-f4a6-4ad2-929f-caacb6346901 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:34.204293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 698/1377] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:36.482322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec6bd-7d43-447e-9f34-b6162257d8b3 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.483183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec6bd-7d43-447e-9f34-b6162257d8b3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.483621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec6bd-7d43-447e-9f34-b6162257d8b3 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.485175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 680/1378] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:36.490565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67d446-8b3f-460e-8c5d-fa00446ea79b None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.490971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67d446-8b3f-460e-8c5d-fa00446ea79b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.491319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67d446-8b3f-460e-8c5d-fa00446ea79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.509044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-9c67d446-8b3f-460e-8c5d-fa00446ea79b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:36.510364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9c67d446-8b3f-460e-8c5d-fa00446ea79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:36.519218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 699/1379] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:36.564987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ec6bd-7d43-447e-9f34-b6162257d8b3 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.581310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ec6bd-7d43-447e-9f34-b6162257d8b3 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.587639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4a948-0139-4f47-8d66-3673a04bd6d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.588351 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a948-0139-4f47-8d66-3673a04bd6d0 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.588847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a948-0139-4f47-8d66-3673a04bd6d0 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.589233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a948-0139-4f47-8d66-3673a04bd6d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.604042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4a948-0139-4f47-8d66-3673a04bd6d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.605944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4a948-0139-4f47-8d66-3673a04bd6d0 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.616467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 681/1380] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:38.926177 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4745b-924c-420d-b68d-4e6b875b4e0a None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.926707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4745b-924c-420d-b68d-4e6b875b4e0a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.927004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4745b-924c-420d-b68d-4e6b875b4e0a None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.930923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 700/1381] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:38.936096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340976-bd6e-4d43-91d4-f3cf4d9658f3 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.936576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340976-bd6e-4d43-91d4-f3cf4d9658f3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.936970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340976-bd6e-4d43-91d4-f3cf4d9658f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.959023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-96340976-bd6e-4d43-91d4-f3cf4d9658f3 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:38.960222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-96340976-bd6e-4d43-91d4-f3cf4d9658f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:38.968089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 682/1382] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:39.010280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4745b-924c-420d-b68d-4e6b875b4e0a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:39.025128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4745b-924c-420d-b68d-4e6b875b4e0a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:39.030607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed61c56-9b70-433b-825f-f474dac06fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:39.031252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed61c56-9b70-433b-825f-f474dac06fed service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.031541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed61c56-9b70-433b-825f-f474dac06fed service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.031849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed61c56-9b70-433b-825f-f474dac06fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.041129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed61c56-9b70-433b-825f-f474dac06fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:39.042717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed61c56-9b70-433b-825f-f474dac06fed service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:39.053334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 701/1383] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:41.226966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224af2f-579a-4b62-a75f-d0009aad7b70 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:41.227530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224af2f-579a-4b62-a75f-d0009aad7b70 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:41.227530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224af2f-579a-4b62-a75f-d0009aad7b70 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:41.228919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 683/1384] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:41.234092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fca4b-dba5-4a68-bf71-cdb37b9db165 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:41.234526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fca4b-dba5-4a68-bf71-cdb37b9db165 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:41.234662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fca4b-dba5-4a68-bf71-cdb37b9db165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:41.252444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-432fca4b-dba5-4a68-bf71-cdb37b9db165 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:41.253677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-432fca4b-dba5-4a68-bf71-cdb37b9db165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:41.262148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 702/1385] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:41.306631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224af2f-579a-4b62-a75f-d0009aad7b70 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:41.323401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224af2f-579a-4b62-a75f-d0009aad7b70 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:41.330911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66746b-6dbf-4e17-82b7-7218cf4bb064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:41.331574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66746b-6dbf-4e17-82b7-7218cf4bb064 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:41.331861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66746b-6dbf-4e17-82b7-7218cf4bb064 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:41.332140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66746b-6dbf-4e17-82b7-7218cf4bb064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:41.340992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66746b-6dbf-4e17-82b7-7218cf4bb064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:41.342573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66746b-6dbf-4e17-82b7-7218cf4bb064 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:41.352661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 684/1386] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:43.709976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4563e6-b276-4030-b15e-9e6c0ad52fb3 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.710506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4563e6-b276-4030-b15e-9e6c0ad52fb3 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.710621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4563e6-b276-4030-b15e-9e6c0ad52fb3 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.711980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 703/1387] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:43.717674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfba2b0-7759-48d4-82f8-4defc0e267cf None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.718095 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfba2b0-7759-48d4-82f8-4defc0e267cf None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.718460 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfba2b0-7759-48d4-82f8-4defc0e267cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.742076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4dfba2b0-7759-48d4-82f8-4defc0e267cf None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:43.743601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4dfba2b0-7759-48d4-82f8-4defc0e267cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:43.753437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 685/1388] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:43.798948 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4563e6-b276-4030-b15e-9e6c0ad52fb3 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:43.814519 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4563e6-b276-4030-b15e-9e6c0ad52fb3 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:43.821193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ffd6-ab51-452e-8c8e-8abc974a3ed9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:43.821875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ffd6-ab51-452e-8c8e-8abc974a3ed9 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.822180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ffd6-ab51-452e-8c8e-8abc974a3ed9 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.822478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ffd6-ab51-452e-8c8e-8abc974a3ed9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.831459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ffd6-ab51-452e-8c8e-8abc974a3ed9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:43.833179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ffd6-ab51-452e-8c8e-8abc974a3ed9 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:43.843301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 704/1389] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:46.406727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2536dd-e971-42d3-b7be-1c51b6456b42 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.407269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2536dd-e971-42d3-b7be-1c51b6456b42 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.407269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2536dd-e971-42d3-b7be-1c51b6456b42 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.408678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 686/1390] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:46.413950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d714fa3-6b45-4789-9aa7-20c5a73b6f13 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.414385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d714fa3-6b45-4789-9aa7-20c5a73b6f13 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.414522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d714fa3-6b45-4789-9aa7-20c5a73b6f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.431922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5d714fa3-6b45-4789-9aa7-20c5a73b6f13 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:46.433526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5d714fa3-6b45-4789-9aa7-20c5a73b6f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.441240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 705/1391] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:46.484651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2536dd-e971-42d3-b7be-1c51b6456b42 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:46.499227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2536dd-e971-42d3-b7be-1c51b6456b42 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:46.504841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147db5e-5bdf-4a87-89ed-131dd9c2a7ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:46.506010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147db5e-5bdf-4a87-89ed-131dd9c2a7ab service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.506295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147db5e-5bdf-4a87-89ed-131dd9c2a7ab service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.506575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147db5e-5bdf-4a87-89ed-131dd9c2a7ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.514585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147db5e-5bdf-4a87-89ed-131dd9c2a7ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:46.516268 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147db5e-5bdf-4a87-89ed-131dd9c2a7ab service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:46.526162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 687/1392] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:47.677400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494d6a6-1bef-4266-a5f3-b572460e7fe3 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.678000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494d6a6-1bef-4266-a5f3-b572460e7fe3 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.678000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494d6a6-1bef-4266-a5f3-b572460e7fe3 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.679360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 706/1393] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:47.684996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7364f-5cb2-412a-a281-a561a7ee408e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.685418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7364f-5cb2-412a-a281-a561a7ee408e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.685630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7364f-5cb2-412a-a281-a561a7ee408e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.702814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-b9b7364f-5cb2-412a-a281-a561a7ee408e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:47.703983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b9b7364f-5cb2-412a-a281-a561a7ee408e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.712171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 688/1394] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:47.768341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c39123-cbc7-4e11-8482-74b948f76aec None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.768672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c39123-cbc7-4e11-8482-74b948f76aec None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.768953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c39123-cbc7-4e11-8482-74b948f76aec None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.770352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 707/1395] 10.222.0.20 () {54 vars in 962 bytes} [Thu Jun 16 00:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:47.775793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffde32-945b-4976-abe3-1b9f44ea21ac None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.776161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffde32-945b-4976-abe3-1b9f44ea21ac None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.776507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffde32-945b-4976-abe3-1b9f44ea21ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.824575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-90ffde32-945b-4976-abe3-1b9f44ea21ac None None] MFA Rules not processed for user `c958b233374b4013a8bbe9efbed054a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:47.825785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-90ffde32-945b-4976-abe3-1b9f44ea21ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.887487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 689/1396] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Jun 16 00:27:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:47.895130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c39123-cbc7-4e11-8482-74b948f76aec None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.906429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c39123-cbc7-4e11-8482-74b948f76aec None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:47.909980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4705f-9733-4f7b-b3ff-b87beb5cd86a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c958b233374b4013a8bbe9efbed054a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:47.910610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4705f-9733-4f7b-b3ff-b87beb5cd86a service glance] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.910893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4705f-9733-4f7b-b3ff-b87beb5cd86a service glance] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.911208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4705f-9733-4f7b-b3ff-b87beb5cd86a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.920095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4705f-9733-4f7b-b3ff-b87beb5cd86a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:47.921576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4705f-9733-4f7b-b3ff-b87beb5cd86a service glance] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:47.933087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 708/1397] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:48.766981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c3219-e958-4bbe-adf0-ed912c4d7592 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.767568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c3219-e958-4bbe-adf0-ed912c4d7592 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.767568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c3219-e958-4bbe-adf0-ed912c4d7592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.819244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ea4c3219-e958-4bbe-adf0-ed912c4d7592 None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:48.820354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ea4c3219-e958-4bbe-adf0-ed912c4d7592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.831497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cf5bd-5b05-43a8-bf5b-cad9ac55f160 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.831964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cf5bd-5b05-43a8-bf5b-cad9ac55f160 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.832116 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cf5bd-5b05-43a8-bf5b-cad9ac55f160 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.834139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 709/1398] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:27:48.839507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ba10b-5790-4ebb-8072-90405ac78d9c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.839798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ba10b-5790-4ebb-8072-90405ac78d9c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.840474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ba10b-5790-4ebb-8072-90405ac78d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.846565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:48.858553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-102ba10b-5790-4ebb-8072-90405ac78d9c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:48.859846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-102ba10b-5790-4ebb-8072-90405ac78d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.866675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:48.868344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 710/1399] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:48.886179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 690/1400] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:48.916452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ba10b-5790-4ebb-8072-90405ac78d9c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.929051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ba10b-5790-4ebb-8072-90405ac78d9c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.933023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12f5df-3380-4622-8c05-8de5b812b8ef None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.933500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12f5df-3380-4622-8c05-8de5b812b8ef None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.933956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12f5df-3380-4622-8c05-8de5b812b8ef None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.935141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad6394-f328-48f2-8d98-dace17ed4389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.935621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 691/1401] 10.222.0.20 () {54 vars in 942 bytes} [Thu Jun 16 00:27:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:27:48.935853 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad6394-f328-48f2-8d98-dace17ed4389 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.936139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad6394-f328-48f2-8d98-dace17ed4389 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.936423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad6394-f328-48f2-8d98-dace17ed4389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.944477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9f25c-9f48-4ca6-8d9b-2c455da333e2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.944793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9f25c-9f48-4ca6-8d9b-2c455da333e2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.945068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9f25c-9f48-4ca6-8d9b-2c455da333e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.951902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad6394-f328-48f2-8d98-dace17ed4389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.953574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad6394-f328-48f2-8d98-dace17ed4389 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.964731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 711/1402] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:49.001221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-01b9f25c-9f48-4ca6-8d9b-2c455da333e2 None None] MFA Rules not processed for user `645bbe41756749b98232d0e2eb2e678a`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:49.002451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-01b9f25c-9f48-4ca6-8d9b-2c455da333e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:49.075145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 692/1403] 10.222.0.20 () {58 vars in 1039 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:49.084356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad6394-f328-48f2-8d98-dace17ed4389 service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.095354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad6394-f328-48f2-8d98-dace17ed4389 service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.100349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a17aa4-8efc-4373-9d4b-c87dbebbb481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.101011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a17aa4-8efc-4373-9d4b-c87dbebbb481 service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.101293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a17aa4-8efc-4373-9d4b-c87dbebbb481 service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.101576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a17aa4-8efc-4373-9d4b-c87dbebbb481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.109552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a17aa4-8efc-4373-9d4b-c87dbebbb481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.111104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a17aa4-8efc-4373-9d4b-c87dbebbb481 service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.119554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 712/1404] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:27:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:27:51.310802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd80ea-0fd4-4834-a222-67f598350ef4 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:51.311417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd80ea-0fd4-4834-a222-67f598350ef4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:51.311417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd80ea-0fd4-4834-a222-67f598350ef4 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:51.312751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 693/1405] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:51.319831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f109-2269-4ebb-ae5a-3e389249718a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:51.320327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f109-2269-4ebb-ae5a-3e389249718a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:51.320418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f109-2269-4ebb-ae5a-3e389249718a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:51.339225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-0fc7f109-2269-4ebb-ae5a-3e389249718a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:51.340480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0fc7f109-2269-4ebb-ae5a-3e389249718a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:51.350367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 713/1406] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:51.393998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd80ea-0fd4-4834-a222-67f598350ef4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:51.410898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd80ea-0fd4-4834-a222-67f598350ef4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:51.417014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af190b85-1058-4088-9f0f-06cefec20bc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:51.417692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190b85-1058-4088-9f0f-06cefec20bc1 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:51.417981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190b85-1058-4088-9f0f-06cefec20bc1 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:51.418262 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190b85-1058-4088-9f0f-06cefec20bc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:51.427042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af190b85-1058-4088-9f0f-06cefec20bc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:51.428717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af190b85-1058-4088-9f0f-06cefec20bc1 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:51.439612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 694/1407] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:01.337359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=17 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:01.435214 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=56 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:12.613411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e371bb-b32e-4252-99cd-7ab43738ebe5 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:12.613869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e371bb-b32e-4252-99cd-7ab43738ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:12.614004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e371bb-b32e-4252-99cd-7ab43738ebe5 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:12.615404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 714/1408] 10.222.0.20 () {54 vars in 918 bytes} [Thu Jun 16 00:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:12.620312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a18d-5b1b-4b9f-a51b-973cfaed8efd None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:12.620779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a18d-5b1b-4b9f-a51b-973cfaed8efd None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:12.620907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a18d-5b1b-4b9f-a51b-973cfaed8efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:12.638959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-2ac3a18d-5b1b-4b9f-a51b-973cfaed8efd None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:12.640141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2ac3a18d-5b1b-4b9f-a51b-973cfaed8efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:12.649085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 695/1409] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Jun 16 00:28:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:12.663217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd38d00-9477-42c9-8a91-7225a47aa249 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:12.663513 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd38d00-9477-42c9-8a91-7225a47aa249 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:12.663799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd38d00-9477-42c9-8a91-7225a47aa249 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:12.665204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 715/1410] 10.222.0.20 () {54 vars in 918 bytes} [Thu Jun 16 00:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:12.669638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f527aab-8302-40b5-95b6-1a689dd15c71 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:12.669928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f527aab-8302-40b5-95b6-1a689dd15c71 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:12.670209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f527aab-8302-40b5-95b6-1a689dd15c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:12.687622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9f527aab-8302-40b5-95b6-1a689dd15c71 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:12.688815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9f527aab-8302-40b5-95b6-1a689dd15c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:12.698319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 696/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Jun 16 00:28:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:15.277197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c24026-3b21-4020-aa22-81f5f2488ab3 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.277769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c24026-3b21-4020-aa22-81f5f2488ab3 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.277769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c24026-3b21-4020-aa22-81f5f2488ab3 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.279105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 716/1412] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:15.284220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27fce4-6a49-40cb-aa72-edfb3975fd27 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.284690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27fce4-6a49-40cb-aa72-edfb3975fd27 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.284803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27fce4-6a49-40cb-aa72-edfb3975fd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.302759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-0c27fce4-6a49-40cb-aa72-edfb3975fd27 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:15.303939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-0c27fce4-6a49-40cb-aa72-edfb3975fd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:15.311953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 697/1413] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:15.354103 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c24026-3b21-4020-aa22-81f5f2488ab3 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.368303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c24026-3b21-4020-aa22-81f5f2488ab3 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.373893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f11d9b-e0c8-40f8-96bc-277e27bb942c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.374523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f11d9b-e0c8-40f8-96bc-277e27bb942c service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.374810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f11d9b-e0c8-40f8-96bc-277e27bb942c service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.375091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f11d9b-e0c8-40f8-96bc-277e27bb942c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.383215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f11d9b-e0c8-40f8-96bc-277e27bb942c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.384821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f11d9b-e0c8-40f8-96bc-277e27bb942c service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.394230 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 717/1414] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:16.339470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dda69-a3f5-49a1-a5d1-c6ca74dd5e55 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.340053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dda69-a3f5-49a1-a5d1-c6ca74dd5e55 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.340053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dda69-a3f5-49a1-a5d1-c6ca74dd5e55 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.341465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 698/1415] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:16.346250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07ec36-8dc7-45f2-88e5-556737715ab3 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.346651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07ec36-8dc7-45f2-88e5-556737715ab3 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.346813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07ec36-8dc7-45f2-88e5-556737715ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.365373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-bf07ec36-8dc7-45f2-88e5-556737715ab3 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:16.366561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-bf07ec36-8dc7-45f2-88e5-556737715ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.374328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 718/1416] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:16.532002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dda69-a3f5-49a1-a5d1-c6ca74dd5e55 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:16.546724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dda69-a3f5-49a1-a5d1-c6ca74dd5e55 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:16.552666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fc67a-53e5-41fe-b1df-f36b55d3f53e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:16.553296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc67a-53e5-41fe-b1df-f36b55d3f53e service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.553570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc67a-53e5-41fe-b1df-f36b55d3f53e service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.553844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc67a-53e5-41fe-b1df-f36b55d3f53e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.562092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc67a-53e5-41fe-b1df-f36b55d3f53e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:16.563646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc67a-53e5-41fe-b1df-f36b55d3f53e service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:16.572884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 699/1417] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:17.579064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136e6fd-50b1-439d-95f2-0e021ba86a96 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.579624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136e6fd-50b1-439d-95f2-0e021ba86a96 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.579624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136e6fd-50b1-439d-95f2-0e021ba86a96 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.581023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 719/1418] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:17.586748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a6f2-230a-424b-80fa-90f4a3342aca None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.587288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a6f2-230a-424b-80fa-90f4a3342aca None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.587577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a6f2-230a-424b-80fa-90f4a3342aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.605798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-cff6a6f2-230a-424b-80fa-90f4a3342aca None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:17.606941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cff6a6f2-230a-424b-80fa-90f4a3342aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.616012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 700/1419] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:17.775097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136e6fd-50b1-439d-95f2-0e021ba86a96 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.790026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136e6fd-50b1-439d-95f2-0e021ba86a96 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.795457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ee0ed-4e90-4bd9-84b0-41b4abc3ab69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.796098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ee0ed-4e90-4bd9-84b0-41b4abc3ab69 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.796388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ee0ed-4e90-4bd9-84b0-41b4abc3ab69 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.796708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ee0ed-4e90-4bd9-84b0-41b4abc3ab69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.806448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ee0ed-4e90-4bd9-84b0-41b4abc3ab69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.808037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ee0ed-4e90-4bd9-84b0-41b4abc3ab69 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.818539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 720/1420] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:18.726497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349573-3146-4228-9d30-c324fbafbe99 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.727056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349573-3146-4228-9d30-c324fbafbe99 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.727056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349573-3146-4228-9d30-c324fbafbe99 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.728424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 701/1421] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:18.733378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d37ce-050d-41c5-afcf-789a3bfb84c6 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.733819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d37ce-050d-41c5-afcf-789a3bfb84c6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.733953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d37ce-050d-41c5-afcf-789a3bfb84c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.752091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5f1d37ce-050d-41c5-afcf-789a3bfb84c6 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:18.753375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5f1d37ce-050d-41c5-afcf-789a3bfb84c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:18.761134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 721/1422] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:18.911624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73349573-3146-4228-9d30-c324fbafbe99 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:18.925645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73349573-3146-4228-9d30-c324fbafbe99 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:18.931594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8266e1-fc88-49bb-b11e-332b19fc5699 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:18.932226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8266e1-fc88-49bb-b11e-332b19fc5699 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.932506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8266e1-fc88-49bb-b11e-332b19fc5699 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.932828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8266e1-fc88-49bb-b11e-332b19fc5699 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.941145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8266e1-fc88-49bb-b11e-332b19fc5699 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:18.942838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8266e1-fc88-49bb-b11e-332b19fc5699 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:18.952124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 702/1423] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:19.898419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcec40-79e9-4f12-afaa-3c581dfbce09 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:19.898956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcec40-79e9-4f12-afaa-3c581dfbce09 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:19.899058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcec40-79e9-4f12-afaa-3c581dfbce09 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:19.900363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 722/1424] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:19.905196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e188c-97b1-4e1e-882e-9f9f3026d245 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:19.905611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e188c-97b1-4e1e-882e-9f9f3026d245 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:19.905760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e188c-97b1-4e1e-882e-9f9f3026d245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:19.923230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7a9e188c-97b1-4e1e-882e-9f9f3026d245 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:19.924401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7a9e188c-97b1-4e1e-882e-9f9f3026d245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:19.932560 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 703/1425] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:20.084171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcec40-79e9-4f12-afaa-3c581dfbce09 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:20.099010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcec40-79e9-4f12-afaa-3c581dfbce09 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:20.104452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e402f6-b1b8-4cc5-808c-6733b4473463 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:20.105115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e402f6-b1b8-4cc5-808c-6733b4473463 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:20.105403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e402f6-b1b8-4cc5-808c-6733b4473463 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:20.105686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e402f6-b1b8-4cc5-808c-6733b4473463 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:20.113767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e402f6-b1b8-4cc5-808c-6733b4473463 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:20.115942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e402f6-b1b8-4cc5-808c-6733b4473463 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:20.125227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 723/1426] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:21.024446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d878e1-506e-48de-93d2-caa2a25d4725 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:21.025010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d878e1-506e-48de-93d2-caa2a25d4725 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:21.025010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d878e1-506e-48de-93d2-caa2a25d4725 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:21.026423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 704/1427] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:21.031156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce3e45-2c44-464c-a994-b5c6f6434d56 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:21.031577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce3e45-2c44-464c-a994-b5c6f6434d56 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:21.031725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce3e45-2c44-464c-a994-b5c6f6434d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:21.049352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e1ce3e45-2c44-464c-a994-b5c6f6434d56 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:21.060668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e1ce3e45-2c44-464c-a994-b5c6f6434d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:21.060668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 724/1428] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:21.217602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d878e1-506e-48de-93d2-caa2a25d4725 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:21.231987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d878e1-506e-48de-93d2-caa2a25d4725 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:21.237296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ad623-bbdb-443f-87e5-3b35d96aed83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:21.237937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ad623-bbdb-443f-87e5-3b35d96aed83 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:21.238224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ad623-bbdb-443f-87e5-3b35d96aed83 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:21.238500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ad623-bbdb-443f-87e5-3b35d96aed83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:21.247031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ad623-bbdb-443f-87e5-3b35d96aed83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:21.248626 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ad623-bbdb-443f-87e5-3b35d96aed83 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:21.258522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 705/1429] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:22.168545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843423-ed72-40b4-96d6-5ca93d0b0c8c None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:22.169083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843423-ed72-40b4-96d6-5ca93d0b0c8c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:22.169181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843423-ed72-40b4-96d6-5ca93d0b0c8c None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:22.170490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 725/1430] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:22.175356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236b67e-9fe3-4555-a900-fc3179722093 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:22.175732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236b67e-9fe3-4555-a900-fc3179722093 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:22.175926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236b67e-9fe3-4555-a900-fc3179722093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:22.194773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7236b67e-9fe3-4555-a900-fc3179722093 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:22.195964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7236b67e-9fe3-4555-a900-fc3179722093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:22.203696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 706/1431] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:22.358687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843423-ed72-40b4-96d6-5ca93d0b0c8c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:22.372713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843423-ed72-40b4-96d6-5ca93d0b0c8c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:22.377940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567f6a8-dbde-4d71-900c-5de49a2b23f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:22.378575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f6a8-dbde-4d71-900c-5de49a2b23f4 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:22.378859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f6a8-dbde-4d71-900c-5de49a2b23f4 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:22.379140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f6a8-dbde-4d71-900c-5de49a2b23f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:22.387595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567f6a8-dbde-4d71-900c-5de49a2b23f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:22.389213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567f6a8-dbde-4d71-900c-5de49a2b23f4 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:22.399303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 726/1432] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:23.323072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddd962-fef3-44ba-bfa5-7b634753f10e None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:23.323649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddd962-fef3-44ba-bfa5-7b634753f10e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:23.323649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddd962-fef3-44ba-bfa5-7b634753f10e None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:23.325013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 707/1433] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:23.329716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c705530-4a5e-4c52-9458-4e8a5b1c1217 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:23.330143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c705530-4a5e-4c52-9458-4e8a5b1c1217 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:23.330270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c705530-4a5e-4c52-9458-4e8a5b1c1217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:23.349232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6c705530-4a5e-4c52-9458-4e8a5b1c1217 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:23.350422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6c705530-4a5e-4c52-9458-4e8a5b1c1217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:23.358187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 727/1434] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:23.508714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddd962-fef3-44ba-bfa5-7b634753f10e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:23.523193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddd962-fef3-44ba-bfa5-7b634753f10e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:23.528897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffafab1-29a3-4621-a436-7069f8bc16e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:23.529536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffafab1-29a3-4621-a436-7069f8bc16e9 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:23.529848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffafab1-29a3-4621-a436-7069f8bc16e9 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:23.530124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffafab1-29a3-4621-a436-7069f8bc16e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:23.538852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffafab1-29a3-4621-a436-7069f8bc16e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:23.540446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffafab1-29a3-4621-a436-7069f8bc16e9 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:23.549609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 708/1435] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:24.484107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d47680-c9d3-4931-9ffe-1a1259220ded None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:24.484679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d47680-c9d3-4931-9ffe-1a1259220ded None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:24.484679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d47680-c9d3-4931-9ffe-1a1259220ded None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:24.486217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 728/1436] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:24.492341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca88129-f8b5-458f-acc4-c4dd0afe2c5b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:24.492829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca88129-f8b5-458f-acc4-c4dd0afe2c5b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:24.492934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca88129-f8b5-458f-acc4-c4dd0afe2c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:24.511773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-aca88129-f8b5-458f-acc4-c4dd0afe2c5b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:24.513016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-aca88129-f8b5-458f-acc4-c4dd0afe2c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:24.520701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 709/1437] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:24.672550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d47680-c9d3-4931-9ffe-1a1259220ded None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:24.687352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d47680-c9d3-4931-9ffe-1a1259220ded None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:24.692609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72f6a4-7bf0-49f3-a842-3cd55decb060 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:24.693241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f6a4-7bf0-49f3-a842-3cd55decb060 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:24.693527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f6a4-7bf0-49f3-a842-3cd55decb060 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:24.693804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f6a4-7bf0-49f3-a842-3cd55decb060 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:24.701769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72f6a4-7bf0-49f3-a842-3cd55decb060 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:24.703354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72f6a4-7bf0-49f3-a842-3cd55decb060 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:24.712634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 729/1438] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:26.741127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ca299-84c5-4560-bc9c-c9b78995488e None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:26.741642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ca299-84c5-4560-bc9c-c9b78995488e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:26.741767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ca299-84c5-4560-bc9c-c9b78995488e None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:26.743046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 710/1439] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:28:26.748460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62b1a-4081-4a90-965a-c549bfc4f291 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:26.758645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62b1a-4081-4a90-965a-c549bfc4f291 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:26.758645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62b1a-4081-4a90-965a-c549bfc4f291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:26.766620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d3e62b1a-4081-4a90-965a-c549bfc4f291 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:26.767801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d3e62b1a-4081-4a90-965a-c549bfc4f291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:26.776038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 730/1440] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:26.814308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ca299-84c5-4560-bc9c-c9b78995488e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:26.824938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ca299-84c5-4560-bc9c-c9b78995488e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:26.828422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a33623-2ad9-4474-9bbf-bb381a337243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:26.829088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33623-2ad9-4474-9bbf-bb381a337243 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:26.829373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33623-2ad9-4474-9bbf-bb381a337243 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:26.829676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33623-2ad9-4474-9bbf-bb381a337243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:26.839040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a33623-2ad9-4474-9bbf-bb381a337243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:26.840457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a33623-2ad9-4474-9bbf-bb381a337243 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:26.850310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 711/1441] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:29.133345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7d98f-3ea2-45d0-ba5f-d374405eef39 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:29.133896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7d98f-3ea2-45d0-ba5f-d374405eef39 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:29.134025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7d98f-3ea2-45d0-ba5f-d374405eef39 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:29.135359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 731/1442] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:29.140647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a572ae-6a6e-4d1e-a0d0-4d3bd45bfa3a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:29.141132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a572ae-6a6e-4d1e-a0d0-4d3bd45bfa3a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:29.141233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a572ae-6a6e-4d1e-a0d0-4d3bd45bfa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:29.159221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-35a572ae-6a6e-4d1e-a0d0-4d3bd45bfa3a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:29.160403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-35a572ae-6a6e-4d1e-a0d0-4d3bd45bfa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:29.168233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 712/1443] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:29.205368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7d98f-3ea2-45d0-ba5f-d374405eef39 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:29.216499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7d98f-3ea2-45d0-ba5f-d374405eef39 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:29.219944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec65dd8-2d8f-46ab-9cbc-6e60ba7e0f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:29.220602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec65dd8-2d8f-46ab-9cbc-6e60ba7e0f59 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:29.220884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec65dd8-2d8f-46ab-9cbc-6e60ba7e0f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:29.221158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec65dd8-2d8f-46ab-9cbc-6e60ba7e0f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:29.229869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec65dd8-2d8f-46ab-9cbc-6e60ba7e0f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:29.231274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec65dd8-2d8f-46ab-9cbc-6e60ba7e0f59 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:29.240242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 732/1444] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:31.690003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475925-a71c-43cc-879d-6801e2c08aa8 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:31.690540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475925-a71c-43cc-879d-6801e2c08aa8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:31.690540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475925-a71c-43cc-879d-6801e2c08aa8 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:31.691866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 713/1445] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:31.697248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef8b8-2510-4b8c-969e-8e379fa91874 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:31.697719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef8b8-2510-4b8c-969e-8e379fa91874 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:31.697832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef8b8-2510-4b8c-969e-8e379fa91874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:31.714907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-eb3ef8b8-2510-4b8c-969e-8e379fa91874 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:31.716080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-eb3ef8b8-2510-4b8c-969e-8e379fa91874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:31.724129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 733/1446] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:31.761606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed475925-a71c-43cc-879d-6801e2c08aa8 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:31.772708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed475925-a71c-43cc-879d-6801e2c08aa8 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:31.776208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e8baf-bd61-4555-8537-bfce0f5920c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:31.776863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e8baf-bd61-4555-8537-bfce0f5920c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:31.777144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e8baf-bd61-4555-8537-bfce0f5920c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:31.777421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e8baf-bd61-4555-8537-bfce0f5920c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:31.786939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e8baf-bd61-4555-8537-bfce0f5920c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:31.788352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e8baf-bd61-4555-8537-bfce0f5920c3 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:31.797609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 714/1447] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:34.035273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085b043-c0fd-4f1c-b828-2dcf8f53c32d None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:34.035928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085b043-c0fd-4f1c-b828-2dcf8f53c32d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:34.035928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085b043-c0fd-4f1c-b828-2dcf8f53c32d None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:34.037251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 734/1448] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:34.042512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f970-e48a-4d89-b3b6-80c2b27453aa None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:34.042951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f970-e48a-4d89-b3b6-80c2b27453aa None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:34.043058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f970-e48a-4d89-b3b6-80c2b27453aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:34.061550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ca46f970-e48a-4d89-b3b6-80c2b27453aa None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:34.062761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ca46f970-e48a-4d89-b3b6-80c2b27453aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:34.071253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 715/1449] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:34.108923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085b043-c0fd-4f1c-b828-2dcf8f53c32d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:34.120025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085b043-c0fd-4f1c-b828-2dcf8f53c32d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:34.123434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ec48e-fdd3-4c8e-aa53-8715b338e77f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:34.124061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec48e-fdd3-4c8e-aa53-8715b338e77f service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:34.124341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec48e-fdd3-4c8e-aa53-8715b338e77f service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:34.124653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec48e-fdd3-4c8e-aa53-8715b338e77f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:34.134068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ec48e-fdd3-4c8e-aa53-8715b338e77f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:34.135470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ec48e-fdd3-4c8e-aa53-8715b338e77f service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:34.144656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 735/1450] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:36.453935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2de0-20a9-4d44-8eeb-96c7a9241236 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:36.454472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2de0-20a9-4d44-8eeb-96c7a9241236 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:36.454472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2de0-20a9-4d44-8eeb-96c7a9241236 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:36.455862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 716/1451] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:36.461727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea893c-e59f-4d9a-a493-6d2e60b95f4d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:36.462127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea893c-e59f-4d9a-a493-6d2e60b95f4d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:36.462285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea893c-e59f-4d9a-a493-6d2e60b95f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:36.480013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-0bea893c-e59f-4d9a-a493-6d2e60b95f4d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:36.481220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0bea893c-e59f-4d9a-a493-6d2e60b95f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:36.488847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 736/1452] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:36.526100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb2de0-20a9-4d44-8eeb-96c7a9241236 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:36.540549 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb2de0-20a9-4d44-8eeb-96c7a9241236 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:36.545946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff03687-b0d4-4e24-8112-c62486353a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:36.546595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff03687-b0d4-4e24-8112-c62486353a95 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:36.546905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff03687-b0d4-4e24-8112-c62486353a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:36.547185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff03687-b0d4-4e24-8112-c62486353a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:36.559313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff03687-b0d4-4e24-8112-c62486353a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:36.560772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff03687-b0d4-4e24-8112-c62486353a95 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:36.572417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 717/1453] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:38.741031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80181e08-7024-404e-973b-bc61ba648de5 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:38.741598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80181e08-7024-404e-973b-bc61ba648de5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:38.741598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80181e08-7024-404e-973b-bc61ba648de5 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:38.743108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 737/1454] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:38.748161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf64291-19bf-4378-85cc-3fb36762350b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:38.748602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf64291-19bf-4378-85cc-3fb36762350b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:38.748759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf64291-19bf-4378-85cc-3fb36762350b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:38.766610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-dbf64291-19bf-4378-85cc-3fb36762350b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:38.767809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-dbf64291-19bf-4378-85cc-3fb36762350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:38.775622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 718/1455] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:38.813108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80181e08-7024-404e-973b-bc61ba648de5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:38.824218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80181e08-7024-404e-973b-bc61ba648de5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:38.827642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094de97-f633-4a6c-b7fc-735ed8ab04b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:38.828265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094de97-f633-4a6c-b7fc-735ed8ab04b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:38.828570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094de97-f633-4a6c-b7fc-735ed8ab04b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:38.828851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094de97-f633-4a6c-b7fc-735ed8ab04b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:38.837471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094de97-f633-4a6c-b7fc-735ed8ab04b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:38.838880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094de97-f633-4a6c-b7fc-735ed8ab04b0 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:38.847971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 738/1456] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:41.030238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871794ee-d689-4901-87a8-8ecc0225f2c0 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:41.030808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871794ee-d689-4901-87a8-8ecc0225f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:41.030808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871794ee-d689-4901-87a8-8ecc0225f2c0 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:41.032159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 719/1457] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:41.037554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135e19d-49ea-4cb3-9a08-2131e7649e55 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:41.037982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135e19d-49ea-4cb3-9a08-2131e7649e55 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:41.038121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135e19d-49ea-4cb3-9a08-2131e7649e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:41.055937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4135e19d-49ea-4cb3-9a08-2131e7649e55 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:41.057971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4135e19d-49ea-4cb3-9a08-2131e7649e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:41.065202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 739/1458] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:41.103047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871794ee-d689-4901-87a8-8ecc0225f2c0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:41.113615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871794ee-d689-4901-87a8-8ecc0225f2c0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:41.117652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199d7bb-b7ab-4904-bbd7-df330c5fd2bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:41.118281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199d7bb-b7ab-4904-bbd7-df330c5fd2bc service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:41.118566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199d7bb-b7ab-4904-bbd7-df330c5fd2bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:41.118845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199d7bb-b7ab-4904-bbd7-df330c5fd2bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:41.127525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199d7bb-b7ab-4904-bbd7-df330c5fd2bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:41.129002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199d7bb-b7ab-4904-bbd7-df330c5fd2bc service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:41.138164 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 720/1459] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:43.289988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bba6a6-296e-424b-9b4d-1e44fd30d2ff None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.290557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bba6a6-296e-424b-9b4d-1e44fd30d2ff None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.290557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bba6a6-296e-424b-9b4d-1e44fd30d2ff None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.291917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 740/1460] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:43.297293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fa89d-8709-4e2c-bf86-c5ebc53b6a6c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.297726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fa89d-8709-4e2c-bf86-c5ebc53b6a6c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.297862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fa89d-8709-4e2c-bf86-c5ebc53b6a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.304700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=79 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:43.317477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d46fa89d-8709-4e2c-bf86-c5ebc53b6a6c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:43.318676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d46fa89d-8709-4e2c-bf86-c5ebc53b6a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.326458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 721/1461] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:43.363254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bba6a6-296e-424b-9b4d-1e44fd30d2ff None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.369560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=73 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:43.375419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bba6a6-296e-424b-9b4d-1e44fd30d2ff None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.378984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eb013-f6ec-4876-bead-bbb03e77a0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.379615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eb013-f6ec-4876-bead-bbb03e77a0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.379900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eb013-f6ec-4876-bead-bbb03e77a0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.380181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eb013-f6ec-4876-bead-bbb03e77a0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.388934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eb013-f6ec-4876-bead-bbb03e77a0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.390403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eb013-f6ec-4876-bead-bbb03e77a0b6 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.400205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 741/1462] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:45.561685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cfccc-f3f1-4fa0-8129-df90fad49b79 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.562265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cfccc-f3f1-4fa0-8129-df90fad49b79 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.562265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cfccc-f3f1-4fa0-8129-df90fad49b79 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.563610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 722/1463] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:45.569081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81393c-b7ba-4c8a-822d-0d536554a5eb None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.569489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81393c-b7ba-4c8a-822d-0d536554a5eb None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.569640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81393c-b7ba-4c8a-822d-0d536554a5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.587429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-8b81393c-b7ba-4c8a-822d-0d536554a5eb None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:45.588661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8b81393c-b7ba-4c8a-822d-0d536554a5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.596481 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 742/1464] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:45.633155 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cfccc-f3f1-4fa0-8129-df90fad49b79 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.643781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cfccc-f3f1-4fa0-8129-df90fad49b79 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.647835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cc751-4de0-4b5b-be2e-450a107e1943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.648450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cc751-4de0-4b5b-be2e-450a107e1943 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.648765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cc751-4de0-4b5b-be2e-450a107e1943 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.649043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cc751-4de0-4b5b-be2e-450a107e1943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.657777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3cc751-4de0-4b5b-be2e-450a107e1943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.659181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3cc751-4de0-4b5b-be2e-450a107e1943 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.668384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 723/1465] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:47.782855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4794917-5393-4d2f-8f43-2e72ce0d5db5 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.783377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4794917-5393-4d2f-8f43-2e72ce0d5db5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.783534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4794917-5393-4d2f-8f43-2e72ce0d5db5 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.784956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 743/1466] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:28:47.790744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88333b87-7fff-4ebb-b4b1-3d8bd84cff0b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.791190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88333b87-7fff-4ebb-b4b1-3d8bd84cff0b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.791286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88333b87-7fff-4ebb-b4b1-3d8bd84cff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.808481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-88333b87-7fff-4ebb-b4b1-3d8bd84cff0b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:47.809947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-88333b87-7fff-4ebb-b4b1-3d8bd84cff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.817794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 724/1467] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:47.854340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4794917-5393-4d2f-8f43-2e72ce0d5db5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:47.865177 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4794917-5393-4d2f-8f43-2e72ce0d5db5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:47.868632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717854cd-8bb8-4a4c-af6b-22c431b91468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:47.869265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717854cd-8bb8-4a4c-af6b-22c431b91468 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.869551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717854cd-8bb8-4a4c-af6b-22c431b91468 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.869830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717854cd-8bb8-4a4c-af6b-22c431b91468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.878628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717854cd-8bb8-4a4c-af6b-22c431b91468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:47.880616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717854cd-8bb8-4a4c-af6b-22c431b91468 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:47.889746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 744/1468] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:57.803535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=11 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:57.885709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=14 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:07.849348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035266ea-9beb-448c-b0d5-1dad1b3ddf16 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.849924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035266ea-9beb-448c-b0d5-1dad1b3ddf16 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.849924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035266ea-9beb-448c-b0d5-1dad1b3ddf16 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.851287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 725/1469] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:07.856396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab61c-cae8-4c41-bd80-d608dbed45f0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.856827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab61c-cae8-4c41-bd80-d608dbed45f0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.856986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab61c-cae8-4c41-bd80-d608dbed45f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.874410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6a7ab61c-cae8-4c41-bd80-d608dbed45f0 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:07.875596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6a7ab61c-cae8-4c41-bd80-d608dbed45f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:07.883452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 745/1470] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:07.931170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fb704-8574-48db-9b81-d6a0629ba7dc None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.931471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fb704-8574-48db-9b81-d6a0629ba7dc None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.931761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fb704-8574-48db-9b81-d6a0629ba7dc None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.933177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 726/1471] 10.222.0.20 () {54 vars in 962 bytes} [Thu Jun 16 00:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:07.938078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc6481-16ea-4a78-9ccb-eb6233721d95 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.938369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc6481-16ea-4a78-9ccb-eb6233721d95 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.938652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc6481-16ea-4a78-9ccb-eb6233721d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.956135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e4fc6481-16ea-4a78-9ccb-eb6233721d95 None None] MFA Rules not processed for user `c958b233374b4013a8bbe9efbed054a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:07.957865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e4fc6481-16ea-4a78-9ccb-eb6233721d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:07.964171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 746/1472] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Jun 16 00:29:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:29:07.971267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fb704-8574-48db-9b81-d6a0629ba7dc None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:07.982357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fb704-8574-48db-9b81-d6a0629ba7dc None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.985910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c07bee-65d7-40cb-991c-a1e4a1a46065 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c958b233374b4013a8bbe9efbed054a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.986537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07bee-65d7-40cb-991c-a1e4a1a46065 service glance] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.986826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07bee-65d7-40cb-991c-a1e4a1a46065 service glance] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.987107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07bee-65d7-40cb-991c-a1e4a1a46065 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.995914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c07bee-65d7-40cb-991c-a1e4a1a46065 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.997389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c07bee-65d7-40cb-991c-a1e4a1a46065 service glance] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:08.006822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 727/1473] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:29:10.207512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce40ad-4f3c-4580-bc1b-fedafc8688d9 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:10.208090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce40ad-4f3c-4580-bc1b-fedafc8688d9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:10.208090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce40ad-4f3c-4580-bc1b-fedafc8688d9 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:10.209603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 747/1474] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:10.214966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750023b0-4f27-49e0-857a-0c3d38196495 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:10.215373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750023b0-4f27-49e0-857a-0c3d38196495 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:10.215527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750023b0-4f27-49e0-857a-0c3d38196495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:10.233312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-750023b0-4f27-49e0-857a-0c3d38196495 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:10.234493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-750023b0-4f27-49e0-857a-0c3d38196495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:10.242201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 728/1475] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:10.265186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce40ad-4f3c-4580-bc1b-fedafc8688d9 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:10.278089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce40ad-4f3c-4580-bc1b-fedafc8688d9 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:10.282378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833c082-9827-4656-8e07-e26bc00762a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:10.283017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833c082-9827-4656-8e07-e26bc00762a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:10.283299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833c082-9827-4656-8e07-e26bc00762a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:10.283574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833c082-9827-4656-8e07-e26bc00762a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:10.284307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833c082-9827-4656-8e07-e26bc00762a3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:10.286690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING py.warnings [None req-c833c082-9827-4656-8e07-e26bc00762a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:29:10.286690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: warnings.warn(msg) Jun 16 00:29:10.286690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: Jun 16 00:29:10.287822 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833c082-9827-4656-8e07-e26bc00762a3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:10.306320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 748/1476] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jun 16 00:29:10 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:12.438687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c9c92-5566-468e-8bd7-a48348921d07 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:12.439252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c9c92-5566-468e-8bd7-a48348921d07 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:12.439252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c9c92-5566-468e-8bd7-a48348921d07 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:12.440634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 729/1477] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:12.446402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f51fa-0dad-4803-9b48-f05b9a45b506 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:12.446808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f51fa-0dad-4803-9b48-f05b9a45b506 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:12.446968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f51fa-0dad-4803-9b48-f05b9a45b506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:12.464220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5a8f51fa-0dad-4803-9b48-f05b9a45b506 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:12.465434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5a8f51fa-0dad-4803-9b48-f05b9a45b506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:12.473037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 749/1478] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:12.515311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c9c92-5566-468e-8bd7-a48348921d07 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:12.531136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c9c92-5566-468e-8bd7-a48348921d07 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:12.536590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8d63d-c19a-4b24-87c6-a4f1b5049a7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:12.537227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8d63d-c19a-4b24-87c6-a4f1b5049a7d service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:12.537510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8d63d-c19a-4b24-87c6-a4f1b5049a7d service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:12.537788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8d63d-c19a-4b24-87c6-a4f1b5049a7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:12.546205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8d63d-c19a-4b24-87c6-a4f1b5049a7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:12.547812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8d63d-c19a-4b24-87c6-a4f1b5049a7d service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:12.557827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 730/1479] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:14.844408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660b29-817d-4a64-9beb-d24e68d2bbe0 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.845038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660b29-817d-4a64-9beb-d24e68d2bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.845038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660b29-817d-4a64-9beb-d24e68d2bbe0 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.846411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 750/1480] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:14.852249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c67b-3a59-471f-8b1f-240c0acc736e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.852655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c67b-3a59-471f-8b1f-240c0acc736e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.852861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c67b-3a59-471f-8b1f-240c0acc736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.870160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ef45c67b-3a59-471f-8b1f-240c0acc736e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:14.871327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ef45c67b-3a59-471f-8b1f-240c0acc736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:14.879466 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 731/1481] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:14.930888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925101b9-ae3e-4753-a7f7-afcd48f66f14 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.931779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925101b9-ae3e-4753-a7f7-afcd48f66f14 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.931779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925101b9-ae3e-4753-a7f7-afcd48f66f14 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.933049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 751/1482] 10.222.0.20 () {54 vars in 964 bytes} [Thu Jun 16 00:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:14.939289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d9e02-6fc8-4898-b400-7295845b6dd6 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.939579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d9e02-6fc8-4898-b400-7295845b6dd6 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.939860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d9e02-6fc8-4898-b400-7295845b6dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.976151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-082d9e02-6fc8-4898-b400-7295845b6dd6 None None] MFA Rules not processed for user `96df1ee2cbc14bcc8a8a250fd8f349dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:14.977438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-082d9e02-6fc8-4898-b400-7295845b6dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:14.983938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 732/1483] 10.222.0.20 () {58 vars in 1061 bytes} [Thu Jun 16 00:29:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:14.992952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925101b9-ae3e-4753-a7f7-afcd48f66f14 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.003453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925101b9-ae3e-4753-a7f7-afcd48f66f14 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.007453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3644f92-5566-4377-864d-dd1c93e95e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.008074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3644f92-5566-4377-864d-dd1c93e95e09 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.008355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3644f92-5566-4377-864d-dd1c93e95e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.008659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3644f92-5566-4377-864d-dd1c93e95e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.017507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3644f92-5566-4377-864d-dd1c93e95e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.019006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3644f92-5566-4377-864d-dd1c93e95e09 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.029319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 752/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:17.398044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a6511-9259-48f2-a93f-fc99e37f76a9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:17.398492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a6511-9259-48f2-a93f-fc99e37f76a9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:17.398592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a6511-9259-48f2-a93f-fc99e37f76a9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:17.399986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 733/1485] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:17.405235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b9aa8-b762-42e2-a3d9-026bf4fd66ad None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:17.405651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b9aa8-b762-42e2-a3d9-026bf4fd66ad None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:17.405802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b9aa8-b762-42e2-a3d9-026bf4fd66ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:17.423544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-3e8b9aa8-b762-42e2-a3d9-026bf4fd66ad None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:17.424808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3e8b9aa8-b762-42e2-a3d9-026bf4fd66ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:17.433032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 753/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:17.470011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a6511-9259-48f2-a93f-fc99e37f76a9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:17.481086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a6511-9259-48f2-a93f-fc99e37f76a9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:17.484554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7c887-c0ea-4798-a783-97e865e2832c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:17.485180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c887-c0ea-4798-a783-97e865e2832c service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:17.485464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c887-c0ea-4798-a783-97e865e2832c service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:17.485744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c887-c0ea-4798-a783-97e865e2832c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:17.494997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7c887-c0ea-4798-a783-97e865e2832c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:17.496404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7c887-c0ea-4798-a783-97e865e2832c service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:17.505954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 734/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:19.998979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64a9d-70b5-48b4-8b40-b1864bdd8bda None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:19.999542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64a9d-70b5-48b4-8b40-b1864bdd8bda None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:19.999542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64a9d-70b5-48b4-8b40-b1864bdd8bda None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.000981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 754/1488] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:20.006068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0130a8b-7608-4fb4-a878-6b49b06de739 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.006541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0130a8b-7608-4fb4-a878-6b49b06de739 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.006640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0130a8b-7608-4fb4-a878-6b49b06de739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.024716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c0130a8b-7608-4fb4-a878-6b49b06de739 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:20.025892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c0130a8b-7608-4fb4-a878-6b49b06de739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:20.033547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 735/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:20.073253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe64a9d-70b5-48b4-8b40-b1864bdd8bda None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.084491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe64a9d-70b5-48b4-8b40-b1864bdd8bda None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.087954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b347a-5a4f-476d-a92b-78f55d7cdabf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.088623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b347a-5a4f-476d-a92b-78f55d7cdabf service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.088907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b347a-5a4f-476d-a92b-78f55d7cdabf service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.089186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b347a-5a4f-476d-a92b-78f55d7cdabf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.098644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b347a-5a4f-476d-a92b-78f55d7cdabf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.100225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b347a-5a4f-476d-a92b-78f55d7cdabf service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.109693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 755/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:22.604978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16060-b7a0-4356-9293-d41e7d6d1faf None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:22.605401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16060-b7a0-4356-9293-d41e7d6d1faf None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:22.605569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16060-b7a0-4356-9293-d41e7d6d1faf None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:22.606943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 736/1491] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jun 16 00:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:29:22.612160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83832ba-8cca-47a8-85e0-a8956785eabc None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:22.612593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83832ba-8cca-47a8-85e0-a8956785eabc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:22.612784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83832ba-8cca-47a8-85e0-a8956785eabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:22.630588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c83832ba-8cca-47a8-85e0-a8956785eabc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:22.631771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c83832ba-8cca-47a8-85e0-a8956785eabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:22.639424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 756/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jun 16 00:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:22.677717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e16060-b7a0-4356-9293-d41e7d6d1faf None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:22.688551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e16060-b7a0-4356-9293-d41e7d6d1faf None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:22.692562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea1d18-b278-4356-85bf-68f1ec449142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:22.693193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea1d18-b278-4356-85bf-68f1ec449142 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:22.693476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea1d18-b278-4356-85bf-68f1ec449142 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:22.693751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea1d18-b278-4356-85bf-68f1ec449142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:22.702486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea1d18-b278-4356-85bf-68f1ec449142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:22.703902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea1d18-b278-4356-85bf-68f1ec449142 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:22.713712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 737/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:32.625166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=33 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:32.709699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=42 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:55.869719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0ad00-ef7b-4e26-b6ca-d72e6a9e9e14 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:55.870326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0ad00-ef7b-4e26-b6ca-d72e6a9e9e14 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:55.870326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0ad00-ef7b-4e26-b6ca-d72e6a9e9e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:55.889417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f9e0ad00-ef7b-4e26-b6ca-d72e6a9e9e14 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:55.890610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f9e0ad00-ef7b-4e26-b6ca-d72e6a9e9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:55.898386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 757/1494] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:55.907757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5c7db-78b5-4dd9-92e5-da0589c8386b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:55.908266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5c7db-78b5-4dd9-92e5-da0589c8386b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:55.908381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5c7db-78b5-4dd9-92e5-da0589c8386b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:55.926400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e7f5c7db-78b5-4dd9-92e5-da0589c8386b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:55.927590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e7f5c7db-78b5-4dd9-92e5-da0589c8386b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:55.935565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 738/1495] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:55.940495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0ad00-ef7b-4e26-b6ca-d72e6a9e9e14 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:55.949963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0ad00-ef7b-4e26-b6ca-d72e6a9e9e14 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:55.954375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:55.954998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:55.955274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:55.955546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:55.956260 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:30:55.956586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:55.957780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING py.warnings [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:30:55.957780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: warnings.warn(msg) Jun 16 00:30:55.957780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: Jun 16 00:30:55.958667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:55.969881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 758/1496] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:30:55.977905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5c7db-78b5-4dd9-92e5-da0589c8386b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:55.987478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5c7db-78b5-4dd9-92e5-da0589c8386b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:55.992138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:55.992826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:55.993112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:55.993394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:55.994080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:55.995842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WARNING py.warnings [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:30:55.995842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: warnings.warn(msg) Jun 16 00:30:55.995842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: Jun 16 00:30:55.997071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:56.135386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 739/1497] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:30:55 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:30:56.139682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:56.149689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9b0f-0d2c-4e83-af1f-8b8273ed6ed3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:56.205092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:56.205754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.206034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.206309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.207043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:56.208036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING py.warnings [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:30:56.208036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: warnings.warn(msg) Jun 16 00:30:56.208036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: Jun 16 00:30:56.209250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:56.276710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 759/1498] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:30:56 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:30:56.280812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:56.291401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef3360-64c8-4527-ac18-be5bfd758efd admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:56.295843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:56.296489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.296802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.297082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.297802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:56.299340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WARNING py.warnings [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:30:56.299340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: warnings.warn(msg) Jun 16 00:30:56.299340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: Jun 16 00:30:56.300269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:56.312944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 740/1499] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:30:56.317128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:56.327093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a699a9-fccf-46ab-9e93-35e9b5c0c045 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:56.331708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60dd49-01a4-4101-ae3e-f6cfdf47e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:56.332360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60dd49-01a4-4101-ae3e-f6cfdf47e83a admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.332701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60dd49-01a4-4101-ae3e-f6cfdf47e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.332987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60dd49-01a4-4101-ae3e-f6cfdf47e83a admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.333744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60dd49-01a4-4101-ae3e-f6cfdf47e83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:56.334745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING py.warnings [None req-bd60dd49-01a4-4101-ae3e-f6cfdf47e83a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:30:56.334745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: warnings.warn(msg) Jun 16 00:30:56.334745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: Jun 16 00:30:56.335647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60dd49-01a4-4101-ae3e-f6cfdf47e83a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:56.349292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 760/1500] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:30:56.353263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:56.363470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db08cc0-90ed-4278-8e78-b7a6ed543650 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:56.369946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4e8c0-988b-4f9b-8cb7-113d80e5905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:56.370825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4e8c0-988b-4f9b-8cb7-113d80e5905f admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.371109 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4e8c0-988b-4f9b-8cb7-113d80e5905f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.371387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4e8c0-988b-4f9b-8cb7-113d80e5905f admin admin] PATH_INFO: `/v3/projects/b1e098a1be734df3b31c67f776cfc4db/users/733d985111f1445baa6e526d8d1a3679/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.386160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4e8c0-988b-4f9b-8cb7-113d80e5905f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e098a1be734df3b31c67f776cfc4db, user_id=733d985111f1445baa6e526d8d1a3679, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:56.387120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WARNING py.warnings [None req-52a4e8c0-988b-4f9b-8cb7-113d80e5905f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:30:56.387120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: warnings.warn(msg) Jun 16 00:30:56.387120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: Jun 16 00:30:56.389083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4e8c0-988b-4f9b-8cb7-113d80e5905f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:56.476960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 741/1501] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:30:56 2022] PUT /identity/v3/projects/b1e098a1be734df3b31c67f776cfc4db/users/733d985111f1445baa6e526d8d1a3679/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:56.488994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635934b6-9b9c-410d-9ec1-5ad0d3dd8988 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.489287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635934b6-9b9c-410d-9ec1-5ad0d3dd8988 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.489574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635934b6-9b9c-410d-9ec1-5ad0d3dd8988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.514723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-635934b6-9b9c-410d-9ec1-5ad0d3dd8988 None None] MFA Rules not processed for user `733d985111f1445baa6e526d8d1a3679`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:56.515930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-635934b6-9b9c-410d-9ec1-5ad0d3dd8988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:56.578220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 761/1502] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:30:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:56.592135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806b76a-d6ab-49ac-8242-9c00cbb53f4a None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.592426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806b76a-d6ab-49ac-8242-9c00cbb53f4a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.592737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806b76a-d6ab-49ac-8242-9c00cbb53f4a None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.594097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 742/1503] 10.222.0.20 () {54 vars in 963 bytes} [Thu Jun 16 00:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:30:56.600549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd21f3-bf83-4993-b099-6ab68d49f27e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.600843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd21f3-bf83-4993-b099-6ab68d49f27e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.601121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd21f3-bf83-4993-b099-6ab68d49f27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.618355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a0dd21f3-bf83-4993-b099-6ab68d49f27e None None] MFA Rules not processed for user `fff82e4b1a4248fcbb3566bb7bc84d26`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:56.619524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a0dd21f3-bf83-4993-b099-6ab68d49f27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:56.663122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:56.670906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 762/1504] 10.222.0.20 () {58 vars in 1060 bytes} [Thu Jun 16 00:30:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:56.678821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806b76a-d6ab-49ac-8242-9c00cbb53f4a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:56.689769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806b76a-d6ab-49ac-8242-9c00cbb53f4a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:56.693315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3fd7d-4b1f-4a09-8270-2fc1e80de334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:56.693943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3fd7d-4b1f-4a09-8270-2fc1e80de334 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.694225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3fd7d-4b1f-4a09-8270-2fc1e80de334 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.694503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3fd7d-4b1f-4a09-8270-2fc1e80de334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.702724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3fd7d-4b1f-4a09-8270-2fc1e80de334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:56.704128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3fd7d-4b1f-4a09-8270-2fc1e80de334 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:56.714619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 743/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:56.756153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3c26-a388-4ed6-8fde-ce4ad5fa2e99 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.756447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3c26-a388-4ed6-8fde-ce4ad5fa2e99 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.756756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3c26-a388-4ed6-8fde-ce4ad5fa2e99 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.758067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 763/1506] 10.222.0.20 () {52 vars in 829 bytes} [Thu Jun 16 00:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:30:56.761674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3fd7d-4b1f-4a09-8270-2fc1e80de334 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:56.822924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3fd7d-4b1f-4a09-8270-2fc1e80de334 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:56.827384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:56.828244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:56.828540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:56.828837 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] PATH_INFO: `/v3/users/733d985111f1445baa6e526d8d1a3679` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:56.830520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=733d985111f1445baa6e526d8d1a3679)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:56.831517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WARNING py.warnings [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:30:56.831517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: warnings.warn(msg) Jun 16 00:30:56.831517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: Jun 16 00:30:56.832748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:56.998966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733d985111f1445baa6e526d8d1a3679'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.011847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733d985111f1445baa6e526d8d1a3679'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.012567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733d985111f1445baa6e526d8d1a3679'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.013283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733d985111f1445baa6e526d8d1a3679'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.053594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733d985111f1445baa6e526d8d1a3679'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.070689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:57.078357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 744/1507] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:30:56 2022] DELETE /identity/v3/users/733d985111f1445baa6e526d8d1a3679 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:57.083326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6a01-aba2-4e8b-a6df-92b0b3181e6c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:57.083825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6a01-aba2-4e8b-a6df-92b0b3181e6c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:57.083950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6a01-aba2-4e8b-a6df-92b0b3181e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:57.101242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b32b6a01-aba2-4e8b-a6df-92b0b3181e6c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:57.102430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b32b6a01-aba2-4e8b-a6df-92b0b3181e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:57.160247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 764/1508] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:57.170316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:57.171736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:57.227917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5460d56-804e-4d46-8881-20e0219fb051 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:57.231672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b90405-33e5-4a53-b63b-822517ed1a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:57.232321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b90405-33e5-4a53-b63b-822517ed1a18 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:57.232639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b90405-33e5-4a53-b63b-822517ed1a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:57.232920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b90405-33e5-4a53-b63b-822517ed1a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:57.242393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b90405-33e5-4a53-b63b-822517ed1a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:57.243806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b90405-33e5-4a53-b63b-822517ed1a18 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:57.253653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 745/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:57.684232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b6a01-aba2-4e8b-a6df-92b0b3181e6c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:57.685961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b32b6a01-aba2-4e8b-a6df-92b0b3181e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:57.699552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b6a01-aba2-4e8b-a6df-92b0b3181e6c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:57.704258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:57.705158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:57.705472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:57.705744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] PATH_INFO: `/v3/projects/b1e098a1be734df3b31c67f776cfc4db` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:57.707006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e098a1be734df3b31c67f776cfc4db)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:57.708007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING py.warnings [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:30:57.708007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: warnings.warn(msg) Jun 16 00:30:57.708007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: Jun 16 00:30:57.709247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:57.825878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e098a1be734df3b31c67f776cfc4db', 'request_id': 'req-6f77c918-3707-41dd-965a-9629d493d923'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.905489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e098a1be734df3b31c67f776cfc4db', 'request_id': 'req-6f77c918-3707-41dd-965a-9629d493d923'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.911955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e098a1be734df3b31c67f776cfc4db', 'request_id': 'req-6f77c918-3707-41dd-965a-9629d493d923'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.929964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] The token cache is being invalidate because project b1e098a1be734df3b31c67f776cfc4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:30:57.930249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.930976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f77c918-3707-41dd-965a-9629d493d923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:57.932560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 765/1510] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:30:57 2022] DELETE /identity/v3/projects/b1e098a1be734df3b31c67f776cfc4db => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:31:07.248940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:31:07.927539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:35.151590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a529e9-03c6-40bd-9fda-300744f64af8 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.152190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a529e9-03c6-40bd-9fda-300744f64af8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.152312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a529e9-03c6-40bd-9fda-300744f64af8 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.153745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:32:35.158504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e9a1-b4d9-4394-acc7-5250af4c48e7 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.158970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e9a1-b4d9-4394-acc7-5250af4c48e7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.159071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e9a1-b4d9-4394-acc7-5250af4c48e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.179155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4661e9a1-b4d9-4394-acc7-5250af4c48e7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:35.180285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4661e9a1-b4d9-4394-acc7-5250af4c48e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:35.238208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:35.412782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a529e9-03c6-40bd-9fda-300744f64af8 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:35.414500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-76a529e9-03c6-40bd-9fda-300744f64af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:35.488554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a529e9-03c6-40bd-9fda-300744f64af8 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:35.494407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b060e9-84ad-4457-8cfe-e0bc339c48e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:35.495047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b060e9-84ad-4457-8cfe-e0bc339c48e2 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.495333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b060e9-84ad-4457-8cfe-e0bc339c48e2 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.495612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b060e9-84ad-4457-8cfe-e0bc339c48e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.503608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b060e9-84ad-4457-8cfe-e0bc339c48e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:35.505263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b060e9-84ad-4457-8cfe-e0bc339c48e2 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:35.514617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 747/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:36.406371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1b0e-8a96-4b3e-8368-c074ab4777d7 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:36.406945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1b0e-8a96-4b3e-8368-c074ab4777d7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:36.406945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1b0e-8a96-4b3e-8368-c074ab4777d7 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:36.408314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:32:36.413161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232350d-4256-437c-99e0-2fc8be429ac2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:36.413556 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232350d-4256-437c-99e0-2fc8be429ac2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:36.413729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232350d-4256-437c-99e0-2fc8be429ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:36.432701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f232350d-4256-437c-99e0-2fc8be429ac2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:36.433901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f232350d-4256-437c-99e0-2fc8be429ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:36.441895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:36.593995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba1b0e-8a96-4b3e-8368-c074ab4777d7 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:36.608229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba1b0e-8a96-4b3e-8368-c074ab4777d7 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:36.613654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459568-9ad1-4b96-b86b-7c041ff6a334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:36.614835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459568-9ad1-4b96-b86b-7c041ff6a334 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:36.615123 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459568-9ad1-4b96-b86b-7c041ff6a334 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:36.615403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459568-9ad1-4b96-b86b-7c041ff6a334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:36.623457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4459568-9ad1-4b96-b86b-7c041ff6a334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:36.625077 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4459568-9ad1-4b96-b86b-7c041ff6a334 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:36.635067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 768/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:36.992943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc193b-28a8-4bb1-9733-61ef5ea500d9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:36.993143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc193b-28a8-4bb1-9733-61ef5ea500d9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:36.993414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc193b-28a8-4bb1-9733-61ef5ea500d9 None None] PATH_INFO: `/v3` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:36.994720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 749/1517] 10.222.0.40 () {54 vars in 925 bytes} [Thu Jun 16 00:32:36 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:32:37.000934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ff489-7a6d-455c-b773-9edee5eea05a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:37.001229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ff489-7a6d-455c-b773-9edee5eea05a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:37.001514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ff489-7a6d-455c-b773-9edee5eea05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:37.019213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f62ff489-7a6d-455c-b773-9edee5eea05a None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:37.020423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f62ff489-7a6d-455c-b773-9edee5eea05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:37.082075 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 769/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jun 16 00:32:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:46.222580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941066b-f356-47c6-83a6-bdaabd12deb6 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:46.223079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941066b-f356-47c6-83a6-bdaabd12deb6 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:46.223079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941066b-f356-47c6-83a6-bdaabd12deb6 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:46.224999 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 750/1519] 10.222.0.20 () {54 vars in 960 bytes} [Thu Jun 16 00:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:32:46.230084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103f031-ea87-43a7-9b6c-8db53776940d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:46.230491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103f031-ea87-43a7-9b6c-8db53776940d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:46.230685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103f031-ea87-43a7-9b6c-8db53776940d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:46.248970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-0103f031-ea87-43a7-9b6c-8db53776940d None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:46.250148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0103f031-ea87-43a7-9b6c-8db53776940d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:46.258490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 770/1520] 10.222.0.20 () {58 vars in 1057 bytes} [Thu Jun 16 00:32:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:46.265727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941066b-f356-47c6-83a6-bdaabd12deb6 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:46.279407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941066b-f356-47c6-83a6-bdaabd12deb6 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:46.284342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c822e-74a5-4fbe-b7e7-e84e8ac13b06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:46.285002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c822e-74a5-4fbe-b7e7-e84e8ac13b06 service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:46.285291 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c822e-74a5-4fbe-b7e7-e84e8ac13b06 service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:46.285574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c822e-74a5-4fbe-b7e7-e84e8ac13b06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:46.294347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c822e-74a5-4fbe-b7e7-e84e8ac13b06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:46.295924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c822e-74a5-4fbe-b7e7-e84e8ac13b06 service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:46.305448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 751/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:47.914537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc89d-8d89-4a56-9c77-7e80ded005f9 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:47.915051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc89d-8d89-4a56-9c77-7e80ded005f9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:47.915051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc89d-8d89-4a56-9c77-7e80ded005f9 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:47.916434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:32:47.921329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f781a-0acc-44b7-9c73-27644c61b414 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:47.921759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f781a-0acc-44b7-9c73-27644c61b414 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:47.921893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f781a-0acc-44b7-9c73-27644c61b414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:47.939831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c75f781a-0acc-44b7-9c73-27644c61b414 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:47.941147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c75f781a-0acc-44b7-9c73-27644c61b414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:47.949112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:48.104083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc89d-8d89-4a56-9c77-7e80ded005f9 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:48.118417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc89d-8d89-4a56-9c77-7e80ded005f9 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:48.123817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65716019-2ebc-4e27-8696-6ee9c1220843 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:48.124457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716019-2ebc-4e27-8696-6ee9c1220843 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:48.124775 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716019-2ebc-4e27-8696-6ee9c1220843 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:48.125060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716019-2ebc-4e27-8696-6ee9c1220843 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:48.133658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65716019-2ebc-4e27-8696-6ee9c1220843 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:48.135235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65716019-2ebc-4e27-8696-6ee9c1220843 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:48.144900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 772/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:49.002245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cd646-1edc-482c-8c9f-898deed4e10f None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:49.002767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cd646-1edc-482c-8c9f-898deed4e10f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:49.002767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cd646-1edc-482c-8c9f-898deed4e10f None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:49.004120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:32:49.009285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5934-f67b-4d4f-ad78-6f7156c7a380 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:49.009738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5934-f67b-4d4f-ad78-6f7156c7a380 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:49.009867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5934-f67b-4d4f-ad78-6f7156c7a380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:49.027440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4afb5934-f67b-4d4f-ad78-6f7156c7a380 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:49.028652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4afb5934-f67b-4d4f-ad78-6f7156c7a380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:49.036842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:49.188224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cd646-1edc-482c-8c9f-898deed4e10f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:49.202669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cd646-1edc-482c-8c9f-898deed4e10f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:49.208066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983c27b-f2ba-4fa0-a162-dd39c2e884ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:49.208725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983c27b-f2ba-4fa0-a162-dd39c2e884ba service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:49.209014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983c27b-f2ba-4fa0-a162-dd39c2e884ba service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:49.209292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983c27b-f2ba-4fa0-a162-dd39c2e884ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:49.217302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983c27b-f2ba-4fa0-a162-dd39c2e884ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:49.218885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983c27b-f2ba-4fa0-a162-dd39c2e884ba service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:49.228626 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 754/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:53.535996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5413c-aecb-4c66-9691-15a2e60cc998 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:53.536455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5413c-aecb-4c66-9691-15a2e60cc998 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:53.536625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5413c-aecb-4c66-9691-15a2e60cc998 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:53.537981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:32:53.543315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec9d8-d6bf-4ed7-ba59-c431a7f32cfe None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:53.545373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec9d8-d6bf-4ed7-ba59-c431a7f32cfe None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:53.545373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec9d8-d6bf-4ed7-ba59-c431a7f32cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:53.564364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-77bec9d8-d6bf-4ed7-ba59-c431a7f32cfe None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:53.565612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-77bec9d8-d6bf-4ed7-ba59-c431a7f32cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:53.574304 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:53.612120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5413c-aecb-4c66-9691-15a2e60cc998 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:53.613826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4de5413c-aecb-4c66-9691-15a2e60cc998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:53.670791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5413c-aecb-4c66-9691-15a2e60cc998 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:53.674405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88a90a-2f25-4a7f-b410-ff80196752c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:53.675043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88a90a-2f25-4a7f-b410-ff80196752c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:53.675324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88a90a-2f25-4a7f-b410-ff80196752c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:53.675599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88a90a-2f25-4a7f-b410-ff80196752c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:53.684395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88a90a-2f25-4a7f-b410-ff80196752c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:53.685844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88a90a-2f25-4a7f-b410-ff80196752c8 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:53.690928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:53.696375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 775/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:55.830664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c23d8e-d293-459f-98b2-4a230d0669d6 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:55.831178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c23d8e-d293-459f-98b2-4a230d0669d6 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:55.831178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c23d8e-d293-459f-98b2-4a230d0669d6 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:55.832490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:32:55.838125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6192c-7374-438b-a972-162aca6f0d39 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:55.838534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6192c-7374-438b-a972-162aca6f0d39 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:55.838688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6192c-7374-438b-a972-162aca6f0d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:55.856418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-58b6192c-7374-438b-a972-162aca6f0d39 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:55.857639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-58b6192c-7374-438b-a972-162aca6f0d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:55.865222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:55.901380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c23d8e-d293-459f-98b2-4a230d0669d6 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:55.911825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c23d8e-d293-459f-98b2-4a230d0669d6 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:55.915280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f4cb8-57e9-4e8a-abb5-fbae1d85d838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:55.915903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4cb8-57e9-4e8a-abb5-fbae1d85d838 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:55.916183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4cb8-57e9-4e8a-abb5-fbae1d85d838 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:55.916455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4cb8-57e9-4e8a-abb5-fbae1d85d838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:55.925374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f4cb8-57e9-4e8a-abb5-fbae1d85d838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:55.926840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f4cb8-57e9-4e8a-abb5-fbae1d85d838 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:55.936016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 757/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:59.525327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2d527-d8a5-486a-a58a-40403623c2eb None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:59.525928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2d527-d8a5-486a-a58a-40403623c2eb None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:59.525928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2d527-d8a5-486a-a58a-40403623c2eb None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:59.527287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:32:59.532491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816a5e-d976-47f2-9158-50ab5ac1f739 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:59.532922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816a5e-d976-47f2-9158-50ab5ac1f739 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:59.533081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816a5e-d976-47f2-9158-50ab5ac1f739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:59.539309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=62 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:59.550958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-75816a5e-d976-47f2-9158-50ab5ac1f739 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:59.552160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-75816a5e-d976-47f2-9158-50ab5ac1f739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:59.560289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:59.838985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2d527-d8a5-486a-a58a-40403623c2eb None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:59.840700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-33a2d527-d8a5-486a-a58a-40403623c2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:59.897554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2d527-d8a5-486a-a58a-40403623c2eb None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:59.900992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e28056-45bf-4899-8d88-b4334c8f5a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:59.901633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28056-45bf-4899-8d88-b4334c8f5a68 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:59.901916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28056-45bf-4899-8d88-b4334c8f5a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:59.902192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28056-45bf-4899-8d88-b4334c8f5a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:59.910980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e28056-45bf-4899-8d88-b4334c8f5a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:59.912441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e28056-45bf-4899-8d88-b4334c8f5a68 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:59.924576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 778/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:33:02.078068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437e149-678c-4cc0-bf8d-03cb547711ef None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:02.078664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437e149-678c-4cc0-bf8d-03cb547711ef None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:02.078664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437e149-678c-4cc0-bf8d-03cb547711ef None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:02.079987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:33:02.085304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c006c4c-35d2-4bd3-a6f0-e44a13e71412 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:02.085737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c006c4c-35d2-4bd3-a6f0-e44a13e71412 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:02.085858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c006c4c-35d2-4bd3-a6f0-e44a13e71412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:02.103453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2c006c4c-35d2-4bd3-a6f0-e44a13e71412 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:02.104701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2c006c4c-35d2-4bd3-a6f0-e44a13e71412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:02.112354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:02.377880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437e149-678c-4cc0-bf8d-03cb547711ef None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:02.379596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d437e149-678c-4cc0-bf8d-03cb547711ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:02.392326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437e149-678c-4cc0-bf8d-03cb547711ef None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:02.395838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410b456-bd60-4077-9ac0-3eff6d7295b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:02.396469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b456-bd60-4077-9ac0-3eff6d7295b3 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:02.396780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b456-bd60-4077-9ac0-3eff6d7295b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:02.397060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b456-bd60-4077-9ac0-3eff6d7295b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:02.405770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410b456-bd60-4077-9ac0-3eff6d7295b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:02.407180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410b456-bd60-4077-9ac0-3eff6d7295b3 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:02.417010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 760/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:33:12.098775 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=32 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:12.412923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=8 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:29.299570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7435f-581c-4f13-9bb0-ec0af52a3713 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.300827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7435f-581c-4f13-9bb0-ec0af52a3713 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.300827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7435f-581c-4f13-9bb0-ec0af52a3713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.327044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-76e7435f-581c-4f13-9bb0-ec0af52a3713 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.332652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-76e7435f-581c-4f13-9bb0-ec0af52a3713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.341721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 780/1540] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.353123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb23b6-b99a-4eb8-bafa-d91e3d6e0dfc None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.353740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb23b6-b99a-4eb8-bafa-d91e3d6e0dfc None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.353740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb23b6-b99a-4eb8-bafa-d91e3d6e0dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.368423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ebeb4-12f3-4c62-8d6a-72389bd4902a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.368837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ebeb4-12f3-4c62-8d6a-72389bd4902a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.369130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ebeb4-12f3-4c62-8d6a-72389bd4902a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.378933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d1eb23b6-b99a-4eb8-bafa-d91e3d6e0dfc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.380094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d1eb23b6-b99a-4eb8-bafa-d91e3d6e0dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.394863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-da6ebeb4-12f3-4c62-8d6a-72389bd4902a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.399886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 761/1541] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:29.400006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-da6ebeb4-12f3-4c62-8d6a-72389bd4902a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.400103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb23b6-b99a-4eb8-bafa-d91e3d6e0dfc None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.411258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb23b6-b99a-4eb8-bafa-d91e3d6e0dfc None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.418394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 781/1542] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:29.420819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.421497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.421776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.422047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.422793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:29.423149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.424163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WARNING py.warnings [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:34:29.424163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: warnings.warn(msg) Jun 16 00:34:29.424163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: Jun 16 00:34:29.425702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.429411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0fd2-dd28-4648-a8f3-d5ed6930f983 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.429705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0fd2-dd28-4648-a8f3-d5ed6930f983 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.429999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0fd2-dd28-4648-a8f3-d5ed6930f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.452295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 762/1543] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:34:29.461904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.470047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1d3e0fd2-dd28-4648-a8f3-d5ed6930f983 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.475383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1d3e0fd2-dd28-4648-a8f3-d5ed6930f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.476395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc49f0-d890-4fca-ae34-c40132ec2e20 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.486441 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e38926-d3a4-4441-9fbc-76ae111ce33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.488013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38926-d3a4-4441-9fbc-76ae111ce33d admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.488013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38926-d3a4-4441-9fbc-76ae111ce33d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.488276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38926-d3a4-4441-9fbc-76ae111ce33d admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.489471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e38926-d3a4-4441-9fbc-76ae111ce33d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.491378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e38926-d3a4-4441-9fbc-76ae111ce33d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.495001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 782/1544] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.496422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a288e8-7a76-4147-9004-b4a239fa67c4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.496735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a288e8-7a76-4147-9004-b4a239fa67c4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.497015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a288e8-7a76-4147-9004-b4a239fa67c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.608312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 763/1545] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:29.609721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e38926-d3a4-4441-9fbc-76ae111ce33d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.648941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b6a288e8-7a76-4147-9004-b4a239fa67c4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.650072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b6a288e8-7a76-4147-9004-b4a239fa67c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.676987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e38926-d3a4-4441-9fbc-76ae111ce33d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.681402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1a822-297d-4d38-9c92-1890fb4a48e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.683238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1a822-297d-4d38-9c92-1890fb4a48e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.683238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1a822-297d-4d38-9c92-1890fb4a48e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.683238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1a822-297d-4d38-9c92-1890fb4a48e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.683909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1a822-297d-4d38-9c92-1890fb4a48e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:29.683909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1a822-297d-4d38-9c92-1890fb4a48e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.685416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1a822-297d-4d38-9c92-1890fb4a48e8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.697306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 764/1546] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:29.698661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84cd24-8bf1-44a5-a516-7bdfec30dadc None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.698959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84cd24-8bf1-44a5-a516-7bdfec30dadc None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.699224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84cd24-8bf1-44a5-a516-7bdfec30dadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.716842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-1c84cd24-8bf1-44a5-a516-7bdfec30dadc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.718030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1c84cd24-8bf1-44a5-a516-7bdfec30dadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.721338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 783/1547] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.723080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf1699-667e-4b81-9aa4-5bf4e684f686 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.723365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf1699-667e-4b81-9aa4-5bf4e684f686 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.723639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf1699-667e-4b81-9aa4-5bf4e684f686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.735367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 765/1548] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.736708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d985cc-4528-4d3b-b2c9-3784f7d38152 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.737001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d985cc-4528-4d3b-b2c9-3784f7d38152 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.737834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d985cc-4528-4d3b-b2c9-3784f7d38152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.742403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-11cf1699-667e-4b81-9aa4-5bf4e684f686 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.743844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-11cf1699-667e-4b81-9aa4-5bf4e684f686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.751970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 784/1549] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.758769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf9e3f-c2a9-49d0-80fc-20ed09639884 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.758769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf9e3f-c2a9-49d0-80fc-20ed09639884 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.758769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf9e3f-c2a9-49d0-80fc-20ed09639884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.759089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-b4d985cc-4528-4d3b-b2c9-3784f7d38152 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.759089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b4d985cc-4528-4d3b-b2c9-3784f7d38152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.769519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 766/1550] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.770293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2746ca7-aa19-4020-8c3d-859a0580fb64 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.770631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2746ca7-aa19-4020-8c3d-859a0580fb64 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.770912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2746ca7-aa19-4020-8c3d-859a0580fb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.772073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-dbbf9e3f-c2a9-49d0-80fc-20ed09639884 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.773293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dbbf9e3f-c2a9-49d0-80fc-20ed09639884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.781630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 785/1551] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.782423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf9e3f-c2a9-49d0-80fc-20ed09639884 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.789459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e2746ca7-aa19-4020-8c3d-859a0580fb64 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.790899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e2746ca7-aa19-4020-8c3d-859a0580fb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.792591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf9e3f-c2a9-49d0-80fc-20ed09639884 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.797019 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5ba66-42a3-4f52-9a6f-0bcc19c0fc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.797669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ba66-42a3-4f52-9a6f-0bcc19c0fc13 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.797949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ba66-42a3-4f52-9a6f-0bcc19c0fc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.798221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ba66-42a3-4f52-9a6f-0bcc19c0fc13 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.798927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5ba66-42a3-4f52-9a6f-0bcc19c0fc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.799186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 767/1552] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.802730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2746ca7-aa19-4020-8c3d-859a0580fb64 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.802855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5ba66-42a3-4f52-9a6f-0bcc19c0fc13 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.813305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2746ca7-aa19-4020-8c3d-859a0580fb64 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.817873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd0d32-994f-40a9-b2be-02f048a648d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.818522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0d32-994f-40a9-b2be-02f048a648d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.818802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0d32-994f-40a9-b2be-02f048a648d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.819129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd0d32-994f-40a9-b2be-02f048a648d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.819762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd0d32-994f-40a9-b2be-02f048a648d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.821671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd0d32-994f-40a9-b2be-02f048a648d4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.892253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 786/1553] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:29.894184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a79967-f6b2-4f01-9e31-d45e6bddcde9 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.894476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a79967-f6b2-4f01-9e31-d45e6bddcde9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.894759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a79967-f6b2-4f01-9e31-d45e6bddcde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.912571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-94a79967-f6b2-4f01-9e31-d45e6bddcde9 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.913745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-94a79967-f6b2-4f01-9e31-d45e6bddcde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.921914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 787/1554] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.934875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b571e0-e87f-46e3-810a-1460081d5581 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.934875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b571e0-e87f-46e3-810a-1460081d5581 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.934875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b571e0-e87f-46e3-810a-1460081d5581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.941068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-01b571e0-e87f-46e3-810a-1460081d5581 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.942240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-01b571e0-e87f-46e3-810a-1460081d5581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.950008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 788/1555] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:29.951274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641b08f-6f34-45ca-b581-d6ad373bfacd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.951569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641b08f-6f34-45ca-b581-d6ad373bfacd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.951852 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641b08f-6f34-45ca-b581-d6ad373bfacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.961306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 768/1556] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:29.961306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08464abb-7d76-49ae-8724-7de7404ab0c4 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.961306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08464abb-7d76-49ae-8724-7de7404ab0c4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.961306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08464abb-7d76-49ae-8724-7de7404ab0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.971726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7641b08f-6f34-45ca-b581-d6ad373bfacd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.974043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7641b08f-6f34-45ca-b581-d6ad373bfacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.978766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-08464abb-7d76-49ae-8724-7de7404ab0c4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:29.979982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-08464abb-7d76-49ae-8724-7de7404ab0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.033362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 789/1557] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:30.035514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90331186-e58d-4299-9041-08e8f33c7958 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.035994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90331186-e58d-4299-9041-08e8f33c7958 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.036268 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90331186-e58d-4299-9041-08e8f33c7958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.042479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 769/1558] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:30.043843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b87f8-0b44-4e25-ab1d-d230c2eccbbe None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.044149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b87f8-0b44-4e25-ab1d-d230c2eccbbe None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.044439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b87f8-0b44-4e25-ab1d-d230c2eccbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.057940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-90331186-e58d-4299-9041-08e8f33c7958 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:30.059126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-90331186-e58d-4299-9041-08e8f33c7958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.062673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-cc0b87f8-0b44-4e25-ab1d-d230c2eccbbe None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:30.064115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cc0b87f8-0b44-4e25-ab1d-d230c2eccbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.067115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 790/1559] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:30.067873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90331186-e58d-4299-9041-08e8f33c7958 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.074979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 770/1560] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:30.075624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b87f8-0b44-4e25-ab1d-d230c2eccbbe None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.078786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90331186-e58d-4299-9041-08e8f33c7958 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.082559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.084268 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70212a7c-d759-47a0-b1a7-15553ab66ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.084966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70212a7c-d759-47a0-b1a7-15553ab66ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.085257 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70212a7c-d759-47a0-b1a7-15553ab66ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.085538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70212a7c-d759-47a0-b1a7-15553ab66ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.086267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70212a7c-d759-47a0-b1a7-15553ab66ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.086366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b87f8-0b44-4e25-ab1d-d230c2eccbbe None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.087867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70212a7c-d759-47a0-b1a7-15553ab66ec2 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.090954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.094206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.094206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.094206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.094206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:30.094206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.094776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.098966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 791/1561] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:30.099910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70212a7c-d759-47a0-b1a7-15553ab66ec2 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.114230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 771/1562] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:30.114230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.114506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70212a7c-d759-47a0-b1a7-15553ab66ec2 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.114802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.115453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.115747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.116029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.116783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:30.117095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.118768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.119554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f90ce-e8a7-4f94-b5c9-a994ee884b3d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.125290 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.125944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.126228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.126502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.127179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.128734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WARNING py.warnings [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:34:30.128734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: warnings.warn(msg) Jun 16 00:34:30.128734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: Jun 16 00:34:30.129931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.131987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 792/1563] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:30.133387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.144140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cce68-b7d0-4ae2-8f42-b127c83c5a51 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.148816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.151676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.151676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.151676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.151676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:30.151676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.152714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.154219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.164702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 793/1564] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:30.165418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.174788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a6239-e99e-4e28-8b6e-ec50fa13af27 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.179266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.179897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.180182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.180474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.181209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:30.181549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.183133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.195852 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 794/1565] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:30.196709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.205784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe005f9c-9d90-46d9-946e-383841567ad9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.211108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.211972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.212737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.213197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.213905 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:30.214215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.215764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.218290 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 772/1566] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.220552 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.227292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 795/1567] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:30.228043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.230936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912011bb-56ec-4ecc-b080-e38e6ebd2134 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.236054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.236796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.237087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.237403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.237790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ce7f4-5249-456f-9004-f97c93c1550b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.238145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:30.238461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.240065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.242404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ef967-fbda-45ff-9dbd-f92315b0aaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.247996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ef967-fbda-45ff-9dbd-f92315b0aaad admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.247996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ef967-fbda-45ff-9dbd-f92315b0aaad admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.247996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ef967-fbda-45ff-9dbd-f92315b0aaad admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.247996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ef967-fbda-45ff-9dbd-f92315b0aaad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.247996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ef967-fbda-45ff-9dbd-f92315b0aaad admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.252258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 773/1568] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:30.253961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.258626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 796/1569] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:30.259809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ef967-fbda-45ff-9dbd-f92315b0aaad admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.266464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d5e1d-7637-4779-83c6-85152ac2776f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.270049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ef967-fbda-45ff-9dbd-f92315b0aaad admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.271990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbda9f-bd04-4596-9cc6-a058d129f3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.272807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbda9f-bd04-4596-9cc6-a058d129f3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.277906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbda9f-bd04-4596-9cc6-a058d129f3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.277906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbda9f-bd04-4596-9cc6-a058d129f3c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.277906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbda9f-bd04-4596-9cc6-a058d129f3c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.278202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.278202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.278202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.278202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.278571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbda9f-bd04-4596-9cc6-a058d129f3c4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.278659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.278659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING py.warnings [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:34:30.278659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: warnings.warn(msg) Jun 16 00:34:30.278659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: Jun 16 00:34:30.278659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.369996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 774/1570] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.370796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbda9f-bd04-4596-9cc6-a058d129f3c4 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.422292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 797/1571] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.423183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.435884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbda9f-bd04-4596-9cc6-a058d129f3c4 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.491258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f0bd8-4ad3-43c3-b2cb-19b61cba2780 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.495611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246deba-c4ff-4a62-aba3-991a08072184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.496280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246deba-c4ff-4a62-aba3-991a08072184 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.496593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246deba-c4ff-4a62-aba3-991a08072184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.496995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246deba-c4ff-4a62-aba3-991a08072184 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.497698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246deba-c4ff-4a62-aba3-991a08072184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.499598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246deba-c4ff-4a62-aba3-991a08072184 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.500563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f8f7f-071a-4f8b-aeff-1def66ec0a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.501242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f8f7f-071a-4f8b-aeff-1def66ec0a7e admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.501534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f8f7f-071a-4f8b-aeff-1def66ec0a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.501819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f8f7f-071a-4f8b-aeff-1def66ec0a7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.502514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f8f7f-071a-4f8b-aeff-1def66ec0a7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.505050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f8f7f-071a-4f8b-aeff-1def66ec0a7e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.607687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 798/1572] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 393 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.608542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246deba-c4ff-4a62-aba3-991a08072184 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.639352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 775/1573] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 385 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.639352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f8f7f-071a-4f8b-aeff-1def66ec0a7e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.670464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.672455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246deba-c4ff-4a62-aba3-991a08072184 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.697924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.698308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f8f7f-071a-4f8b-aeff-1def66ec0a7e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.703947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b40fd-21c8-4ba6-b40f-16f98a997a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.704794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b40fd-21c8-4ba6-b40f-16f98a997a60 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.705509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b40fd-21c8-4ba6-b40f-16f98a997a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.706501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b40fd-21c8-4ba6-b40f-16f98a997a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.707232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b40fd-21c8-4ba6-b40f-16f98a997a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.709220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b40fd-21c8-4ba6-b40f-16f98a997a60 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.731631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dd646-c90d-4c49-97d6-be96b1c23bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.732349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dd646-c90d-4c49-97d6-be96b1c23bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.732690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dd646-c90d-4c49-97d6-be96b1c23bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.732956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dd646-c90d-4c49-97d6-be96b1c23bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.733685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dd646-c90d-4c49-97d6-be96b1c23bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.735276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dd646-c90d-4c49-97d6-be96b1c23bd5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.744976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 799/1574] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:30.745726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dd646-c90d-4c49-97d6-be96b1c23bd5 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.755040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dd646-c90d-4c49-97d6-be96b1c23bd5 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.759676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099da29-e0c2-4510-abe5-234bff834b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.760268 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099da29-e0c2-4510-abe5-234bff834b0a admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.760583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099da29-e0c2-4510-abe5-234bff834b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.760920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099da29-e0c2-4510-abe5-234bff834b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.761625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099da29-e0c2-4510-abe5-234bff834b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.763437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099da29-e0c2-4510-abe5-234bff834b0a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.776918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 776/1575] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.776918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b40fd-21c8-4ba6-b40f-16f98a997a60 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.835653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b40fd-21c8-4ba6-b40f-16f98a997a60 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.840467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40bc8e-1f4f-473e-819c-9e7a0f596240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.841416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bc8e-1f4f-473e-819c-9e7a0f596240 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.841700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bc8e-1f4f-473e-819c-9e7a0f596240 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.841981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bc8e-1f4f-473e-819c-9e7a0f596240 admin admin] PATH_INFO: `/v3/projects/17623d8aa6b54f56a5e8e94d10c93bb7/users/b6dee5e53bba4b3ba19b24cdfc3ae53c/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.849881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 800/1576] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.849881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099da29-e0c2-4510-abe5-234bff834b0a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.864001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40bc8e-1f4f-473e-819c-9e7a0f596240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17623d8aa6b54f56a5e8e94d10c93bb7, user_id=b6dee5e53bba4b3ba19b24cdfc3ae53c, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.867083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40bc8e-1f4f-473e-819c-9e7a0f596240 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.915356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099da29-e0c2-4510-abe5-234bff834b0a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.919984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d624b-8480-478f-adf7-34dcbe22f5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.920691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d624b-8480-478f-adf7-34dcbe22f5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.921431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d624b-8480-478f-adf7-34dcbe22f5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.921761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d624b-8480-478f-adf7-34dcbe22f5e5 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.922448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d624b-8480-478f-adf7-34dcbe22f5e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.924419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d624b-8480-478f-adf7-34dcbe22f5e5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.947399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 777/1577] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:30 2022] PUT /identity/v3/projects/17623d8aa6b54f56a5e8e94d10c93bb7/users/b6dee5e53bba4b3ba19b24cdfc3ae53c/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:30.948648 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40bc8e-1f4f-473e-819c-9e7a0f596240 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.010815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40bc8e-1f4f-473e-819c-9e7a0f596240 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.015533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a1b81-0ce7-4b17-8193-57b3bd4b76d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.016246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1b81-0ce7-4b17-8193-57b3bd4b76d0 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.016538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1b81-0ce7-4b17-8193-57b3bd4b76d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.016848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1b81-0ce7-4b17-8193-57b3bd4b76d0 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.017532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a1b81-0ce7-4b17-8193-57b3bd4b76d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.019463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a1b81-0ce7-4b17-8193-57b3bd4b76d0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.026914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 801/1578] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.027795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d624b-8480-478f-adf7-34dcbe22f5e5 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.037963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d624b-8480-478f-adf7-34dcbe22f5e5 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.043922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22b667-4fc3-4358-8cda-fc83a0a32791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.044597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b667-4fc3-4358-8cda-fc83a0a32791 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.044880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b667-4fc3-4358-8cda-fc83a0a32791 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.045179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b667-4fc3-4358-8cda-fc83a0a32791 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.045854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22b667-4fc3-4358-8cda-fc83a0a32791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.047843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22b667-4fc3-4358-8cda-fc83a0a32791 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.118699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 778/1579] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.119636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a1b81-0ce7-4b17-8193-57b3bd4b76d0 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.129177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a1b81-0ce7-4b17-8193-57b3bd4b76d0 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.133655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711c230-91cd-40e7-b57e-d320af76ce13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.134284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711c230-91cd-40e7-b57e-d320af76ce13 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.134569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711c230-91cd-40e7-b57e-d320af76ce13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.134845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711c230-91cd-40e7-b57e-d320af76ce13 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.135511 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711c230-91cd-40e7-b57e-d320af76ce13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.137458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711c230-91cd-40e7-b57e-d320af76ce13 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.147097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 802/1580] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.147983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22b667-4fc3-4358-8cda-fc83a0a32791 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.157676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22b667-4fc3-4358-8cda-fc83a0a32791 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.162820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642b8f9-fd96-4e4f-ba3c-0f0a4c49337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.163484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b8f9-fd96-4e4f-ba3c-0f0a4c49337d admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.163771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b8f9-fd96-4e4f-ba3c-0f0a4c49337d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.164054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b8f9-fd96-4e4f-ba3c-0f0a4c49337d admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.164816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642b8f9-fd96-4e4f-ba3c-0f0a4c49337d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.166393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642b8f9-fd96-4e4f-ba3c-0f0a4c49337d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.177343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 803/1581] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.178063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642b8f9-fd96-4e4f-ba3c-0f0a4c49337d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.187209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642b8f9-fd96-4e4f-ba3c-0f0a4c49337d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.192067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a4dba-eeab-4030-899f-5af9a1afd8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.192768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a4dba-eeab-4030-899f-5af9a1afd8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.193067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a4dba-eeab-4030-899f-5af9a1afd8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.193328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a4dba-eeab-4030-899f-5af9a1afd8f6 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.193985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a4dba-eeab-4030-899f-5af9a1afd8f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.195826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a4dba-eeab-4030-899f-5af9a1afd8f6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.223873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 779/1582] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.224850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711c230-91cd-40e7-b57e-d320af76ce13 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.234236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711c230-91cd-40e7-b57e-d320af76ce13 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.239316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78528fe-0389-4613-ab0c-e0b1c3d41dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.239947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78528fe-0389-4613-ab0c-e0b1c3d41dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.240235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78528fe-0389-4613-ab0c-e0b1c3d41dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.240546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78528fe-0389-4613-ab0c-e0b1c3d41dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.241215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78528fe-0389-4613-ab0c-e0b1c3d41dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.243104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78528fe-0389-4613-ab0c-e0b1c3d41dd8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.251627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.277966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 804/1583] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.277966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5978e-4c9b-49b7-9df3-093041955a6c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.277966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5978e-4c9b-49b7-9df3-093041955a6c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.277966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5978e-4c9b-49b7-9df3-093041955a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.302508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-08b5978e-4c9b-49b7-9df3-093041955a6c None None] MFA Rules not processed for user `b6dee5e53bba4b3ba19b24cdfc3ae53c`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:31.303710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-08b5978e-4c9b-49b7-9df3-093041955a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:31.335282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.366287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 805/1584] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:31.367235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5978e-4c9b-49b7-9df3-093041955a6c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.378040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 780/1585] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.378522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5978e-4c9b-49b7-9df3-093041955a6c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.378911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78528fe-0389-4613-ab0c-e0b1c3d41dd8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.384694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ca4f3-e64c-401d-bae5-773fc93b7558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.385367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ca4f3-e64c-401d-bae5-773fc93b7558 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.385654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ca4f3-e64c-401d-bae5-773fc93b7558 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.385933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ca4f3-e64c-401d-bae5-773fc93b7558 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.386674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ca4f3-e64c-401d-bae5-773fc93b7558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.388346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ca4f3-e64c-401d-bae5-773fc93b7558 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.389913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78528fe-0389-4613-ab0c-e0b1c3d41dd8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.395253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e280382-3d36-4af5-9280-9bcf38c108f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.399938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e280382-3d36-4af5-9280-9bcf38c108f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.399938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e280382-3d36-4af5-9280-9bcf38c108f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.399938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e280382-3d36-4af5-9280-9bcf38c108f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.399938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e280382-3d36-4af5-9280-9bcf38c108f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.399938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e280382-3d36-4af5-9280-9bcf38c108f8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.400473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 806/1586] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.401060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ca4f3-e64c-401d-bae5-773fc93b7558 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.410706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 781/1587] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.411210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ca4f3-e64c-401d-bae5-773fc93b7558 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.411697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e280382-3d36-4af5-9280-9bcf38c108f8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.416298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2a95a-1159-466f-8d2c-0cdd725d1e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.416989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2a95a-1159-466f-8d2c-0cdd725d1e75 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.417277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2a95a-1159-466f-8d2c-0cdd725d1e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.417552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2a95a-1159-466f-8d2c-0cdd725d1e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.418272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2a95a-1159-466f-8d2c-0cdd725d1e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.419903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2a95a-1159-466f-8d2c-0cdd725d1e75 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.421918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e280382-3d36-4af5-9280-9bcf38c108f8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.426853 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f246018-4a11-41cc-8312-9965dee2f6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.427807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f246018-4a11-41cc-8312-9965dee2f6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.428033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f246018-4a11-41cc-8312-9965dee2f6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.428312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f246018-4a11-41cc-8312-9965dee2f6cd admin admin] PATH_INFO: `/v3/projects/45031128a55e42989da61e01948538d3/users/052ae1db33014bb5915dc228f1b76d5c/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.430299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 807/1588] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.431142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2a95a-1159-466f-8d2c-0cdd725d1e75 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.440814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2a95a-1159-466f-8d2c-0cdd725d1e75 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.444766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f246018-4a11-41cc-8312-9965dee2f6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=45031128a55e42989da61e01948538d3, user_id=052ae1db33014bb5915dc228f1b76d5c, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.446110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05936-2844-4259-9c44-cc20f89fe4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.446763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05936-2844-4259-9c44-cc20f89fe4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.447050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05936-2844-4259-9c44-cc20f89fe4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.447333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05936-2844-4259-9c44-cc20f89fe4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.447446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f246018-4a11-41cc-8312-9965dee2f6cd admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.448089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b05936-2844-4259-9c44-cc20f89fe4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.449743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b05936-2844-4259-9c44-cc20f89fe4f4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.460917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 808/1589] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.461770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05936-2844-4259-9c44-cc20f89fe4f4 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.471026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05936-2844-4259-9c44-cc20f89fe4f4 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.476695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5940-61aa-4d6d-afd5-1f449bc4a5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.482121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5940-61aa-4d6d-afd5-1f449bc4a5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.482121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5940-61aa-4d6d-afd5-1f449bc4a5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.482121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5940-61aa-4d6d-afd5-1f449bc4a5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.482121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d5940-61aa-4d6d-afd5-1f449bc4a5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.482121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d5940-61aa-4d6d-afd5-1f449bc4a5c9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.490050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 809/1590] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.490806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5940-61aa-4d6d-afd5-1f449bc4a5c9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.549809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5940-61aa-4d6d-afd5-1f449bc4a5c9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.553581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1073-b275-4499-8725-4652225bc2af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.554217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1073-b275-4499-8725-4652225bc2af service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.554501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1073-b275-4499-8725-4652225bc2af service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.554780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1073-b275-4499-8725-4652225bc2af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.563494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed1073-b275-4499-8725-4652225bc2af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.564983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed1073-b275-4499-8725-4652225bc2af service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.566719 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 782/1591] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/45031128a55e42989da61e01948538d3/users/052ae1db33014bb5915dc228f1b76d5c/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:31.567555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f246018-4a11-41cc-8312-9965dee2f6cd admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.575012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 810/1592] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:31.575579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1073-b275-4499-8725-4652225bc2af service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.590797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1073-b275-4499-8725-4652225bc2af service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.638337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f246018-4a11-41cc-8312-9965dee2f6cd admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.643294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e7bb-20d1-4c18-a0ab-ecff46533d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.644556 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e7bb-20d1-4c18-a0ab-ecff46533d95 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.644840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e7bb-20d1-4c18-a0ab-ecff46533d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.645120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e7bb-20d1-4c18-a0ab-ecff46533d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.645907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e7bb-20d1-4c18-a0ab-ecff46533d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.647508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e7bb-20d1-4c18-a0ab-ecff46533d95 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.658293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 783/1593] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.658528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136719-857a-47c6-8ab7-c93af8a53c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.659371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136719-857a-47c6-8ab7-c93af8a53c86 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.659660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136719-857a-47c6-8ab7-c93af8a53c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.659937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136719-857a-47c6-8ab7-c93af8a53c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.660708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21136719-857a-47c6-8ab7-c93af8a53c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.661978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e7bb-20d1-4c18-a0ab-ecff46533d95 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.662950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21136719-857a-47c6-8ab7-c93af8a53c86 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.672346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e7bb-20d1-4c18-a0ab-ecff46533d95 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.675327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 811/1594] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.676334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136719-857a-47c6-8ab7-c93af8a53c86 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.680340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0d615-b497-4b1b-8392-511b523d587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.681002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0d615-b497-4b1b-8392-511b523d587c admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.681426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0d615-b497-4b1b-8392-511b523d587c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.681711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0d615-b497-4b1b-8392-511b523d587c admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.682465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0d615-b497-4b1b-8392-511b523d587c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.684035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0d615-b497-4b1b-8392-511b523d587c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.689681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136719-857a-47c6-8ab7-c93af8a53c86 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.694674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e604c-1663-45b7-9a2f-e56112ac46ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.699578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 784/1595] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.699744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e604c-1663-45b7-9a2f-e56112ac46ef admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.699744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e604c-1663-45b7-9a2f-e56112ac46ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.699744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e604c-1663-45b7-9a2f-e56112ac46ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.700034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0d615-b497-4b1b-8392-511b523d587c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.700135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e604c-1663-45b7-9a2f-e56112ac46ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.700135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e604c-1663-45b7-9a2f-e56112ac46ef admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.707682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0d615-b497-4b1b-8392-511b523d587c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.710616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 812/1596] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.712376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401ade2-6ecd-488b-91f8-03af863d3e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.713328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ade2-6ecd-488b-91f8-03af863d3e02 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.713775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ade2-6ecd-488b-91f8-03af863d3e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.713897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ade2-6ecd-488b-91f8-03af863d3e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.715729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401ade2-6ecd-488b-91f8-03af863d3e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.715864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e604c-1663-45b7-9a2f-e56112ac46ef admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.716233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401ade2-6ecd-488b-91f8-03af863d3e02 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.724492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e604c-1663-45b7-9a2f-e56112ac46ef admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.727870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 785/1597] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.729331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054510a3-74ac-46df-b51b-a85f93d8f1a3 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.729792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054510a3-74ac-46df-b51b-a85f93d8f1a3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.730234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054510a3-74ac-46df-b51b-a85f93d8f1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.730505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8426bdc-110d-4207-9c45-7939787d6df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.731581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8426bdc-110d-4207-9c45-7939787d6df6 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.732085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8426bdc-110d-4207-9c45-7939787d6df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.732499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8426bdc-110d-4207-9c45-7939787d6df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.733844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8426bdc-110d-4207-9c45-7939787d6df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.736291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8426bdc-110d-4207-9c45-7939787d6df6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.750285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 813/1598] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.750448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-054510a3-74ac-46df-b51b-a85f93d8f1a3 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:31.750551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8426bdc-110d-4207-9c45-7939787d6df6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.750664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-054510a3-74ac-46df-b51b-a85f93d8f1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:31.758049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 786/1599] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:31.759114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054510a3-74ac-46df-b51b-a85f93d8f1a3 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.767679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8426bdc-110d-4207-9c45-7939787d6df6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.769607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054510a3-74ac-46df-b51b-a85f93d8f1a3 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.771248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35f10-aeae-4f2a-9a5a-19dfc16add1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.772212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35f10-aeae-4f2a-9a5a-19dfc16add1d admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.772501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35f10-aeae-4f2a-9a5a-19dfc16add1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.772807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35f10-aeae-4f2a-9a5a-19dfc16add1d admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.773716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c35f10-aeae-4f2a-9a5a-19dfc16add1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.774720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71989b-844a-443e-8988-4de14c6ba0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.775345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c35f10-aeae-4f2a-9a5a-19dfc16add1d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.775609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71989b-844a-443e-8988-4de14c6ba0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.775894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71989b-844a-443e-8988-4de14c6ba0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.776170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71989b-844a-443e-8988-4de14c6ba0f1 admin admin] PATH_INFO: `/v3/projects/a17db5f4c7b74c2f89cb295ce6cd8d80/users/8166ce98a4d94d10aa6c28e84fa93fba/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.776984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.791221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 814/1600] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.791221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35f10-aeae-4f2a-9a5a-19dfc16add1d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.796101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71989b-844a-443e-8988-4de14c6ba0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17db5f4c7b74c2f89cb295ce6cd8d80, user_id=8166ce98a4d94d10aa6c28e84fa93fba, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.796101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71989b-844a-443e-8988-4de14c6ba0f1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.799159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35f10-aeae-4f2a-9a5a-19dfc16add1d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.804403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.805448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.805730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.806116 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] PATH_INFO: `/v3/projects/2d9a178dbd5e430f874fa216367424dc/users/704b1cb48c7249d69ab69172d1a157b1/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.821388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9a178dbd5e430f874fa216367424dc, user_id=704b1cb48c7249d69ab69172d1a157b1, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.830676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING py.warnings [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:34:31.830676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: warnings.warn(msg) Jun 16 00:34:31.830676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: Jun 16 00:34:31.830676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.886329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 787/1601] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/a17db5f4c7b74c2f89cb295ce6cd8d80/users/8166ce98a4d94d10aa6c28e84fa93fba/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:31.887166 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71989b-844a-443e-8988-4de14c6ba0f1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.915652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 815/1602] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/2d9a178dbd5e430f874fa216367424dc/users/704b1cb48c7249d69ab69172d1a157b1/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:31.916690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.953419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71989b-844a-443e-8988-4de14c6ba0f1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.958851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.985301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed2c07-0d58-49f5-9b76-251778ab3181 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.990439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baad084-22f8-4177-be4d-0c6c916c1fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.991417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baad084-22f8-4177-be4d-0c6c916c1fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.991701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baad084-22f8-4177-be4d-0c6c916c1fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.991976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baad084-22f8-4177-be4d-0c6c916c1fb7 admin admin] PATH_INFO: `/v3/projects/bfe70fb31ab14040b64268384c5458e9/users/025df89c03f044d0aa7fd0efea9afb53/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.007088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baad084-22f8-4177-be4d-0c6c916c1fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe70fb31ab14040b64268384c5458e9, user_id=025df89c03f044d0aa7fd0efea9afb53, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.009758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baad084-22f8-4177-be4d-0c6c916c1fb7 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.012935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7a7db-079f-485b-bb1b-72d3c6c56696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.013825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7a7db-079f-485b-bb1b-72d3c6c56696 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.015542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7a7db-079f-485b-bb1b-72d3c6c56696 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.015542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7a7db-079f-485b-bb1b-72d3c6c56696 admin admin] PATH_INFO: `/v3/projects/a4e7be42e39248e4b9f5c89e96a6b82e/users/c9dcfb252f0d44a280b3448094883e13/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.028218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7a7db-079f-485b-bb1b-72d3c6c56696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e7be42e39248e4b9f5c89e96a6b82e, user_id=c9dcfb252f0d44a280b3448094883e13, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.030897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7a7db-079f-485b-bb1b-72d3c6c56696 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.126803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 816/1603] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/bfe70fb31ab14040b64268384c5458e9/users/025df89c03f044d0aa7fd0efea9afb53/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:32.127697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baad084-22f8-4177-be4d-0c6c916c1fb7 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.151273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 788/1604] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/a4e7be42e39248e4b9f5c89e96a6b82e/users/c9dcfb252f0d44a280b3448094883e13/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:32.152378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7a7db-079f-485b-bb1b-72d3c6c56696 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.194187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baad084-22f8-4177-be4d-0c6c916c1fb7 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.214178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7a7db-079f-485b-bb1b-72d3c6c56696 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.217738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea70801-ba7e-4e41-a28c-6cbf13380374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.218385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea70801-ba7e-4e41-a28c-6cbf13380374 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.218673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea70801-ba7e-4e41-a28c-6cbf13380374 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.218954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea70801-ba7e-4e41-a28c-6cbf13380374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.254183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0eacd0-1965-431c-bfdd-4fbe4a4e3466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.255848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eacd0-1965-431c-bfdd-4fbe4a4e3466 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.256160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eacd0-1965-431c-bfdd-4fbe4a4e3466 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.256505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eacd0-1965-431c-bfdd-4fbe4a4e3466 admin admin] PATH_INFO: `/v3/projects/494ce289f1774df0833301de8e46ae11/users/31973e5a04c14cfc90ab377eac5f4194/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.260087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:32.271234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0eacd0-1965-431c-bfdd-4fbe4a4e3466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494ce289f1774df0833301de8e46ae11, user_id=31973e5a04c14cfc90ab377eac5f4194, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.274155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0eacd0-1965-431c-bfdd-4fbe4a4e3466 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.281901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea70801-ba7e-4e41-a28c-6cbf13380374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.283383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea70801-ba7e-4e41-a28c-6cbf13380374 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.292799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 789/1605] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:32.293543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea70801-ba7e-4e41-a28c-6cbf13380374 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.302904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea70801-ba7e-4e41-a28c-6cbf13380374 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.308285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359df943-3ad8-4d32-86e2-0ecf8299e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.309203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359df943-3ad8-4d32-86e2-0ecf8299e46d admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.309487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359df943-3ad8-4d32-86e2-0ecf8299e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.309764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359df943-3ad8-4d32-86e2-0ecf8299e46d admin admin] PATH_INFO: `/v3/projects/704cb48b399648e08be940e082da2c3e/users/f697f7f3ddc142cba550e2385b26552f/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.328016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359df943-3ad8-4d32-86e2-0ecf8299e46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=704cb48b399648e08be940e082da2c3e, user_id=f697f7f3ddc142cba550e2385b26552f, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.330731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359df943-3ad8-4d32-86e2-0ecf8299e46d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.418390 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 790/1606] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:32 2022] PUT /identity/v3/projects/704cb48b399648e08be940e082da2c3e/users/f697f7f3ddc142cba550e2385b26552f/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:32.418958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 817/1607] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:32 2022] PUT /identity/v3/projects/494ce289f1774df0833301de8e46ae11/users/31973e5a04c14cfc90ab377eac5f4194/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:32.418958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723202-94c0-49d6-a556-0239766845f2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.418958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723202-94c0-49d6-a556-0239766845f2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.418958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723202-94c0-49d6-a556-0239766845f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.424433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895d8d2-0690-4a88-ae64-07cfd5225294 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.425437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895d8d2-0690-4a88-ae64-07cfd5225294 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.425899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895d8d2-0690-4a88-ae64-07cfd5225294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.443658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d5723202-94c0-49d6-a556-0239766845f2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:32.445278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d5723202-94c0-49d6-a556-0239766845f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.456222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-2895d8d2-0690-4a88-ae64-07cfd5225294 None None] MFA Rules not processed for user `8166ce98a4d94d10aa6c28e84fa93fba`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:32.457639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2895d8d2-0690-4a88-ae64-07cfd5225294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.463201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:32.529618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 818/1608] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:32.531183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cced3-d55c-4b30-b1fb-c29c0487089d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.531501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cced3-d55c-4b30-b1fb-c29c0487089d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.531780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cced3-d55c-4b30-b1fb-c29c0487089d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.552104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 791/1609] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:32.553432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d8617-ad83-4fb9-8e34-19674d933dbb None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.553722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d8617-ad83-4fb9-8e34-19674d933dbb None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.554000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d8617-ad83-4fb9-8e34-19674d933dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.567706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-db1cced3-d55c-4b30-b1fb-c29c0487089d None None] MFA Rules not processed for user `025df89c03f044d0aa7fd0efea9afb53`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:32.575800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-db1cced3-d55c-4b30-b1fb-c29c0487089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.580789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a54d8617-ad83-4fb9-8e34-19674d933dbb None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:32.582567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a54d8617-ad83-4fb9-8e34-19674d933dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.592212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 792/1610] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:32.593802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326ac8f-fd18-4011-853a-ea273920df91 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.594201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326ac8f-fd18-4011-853a-ea273920df91 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.594588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326ac8f-fd18-4011-853a-ea273920df91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.626087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-3326ac8f-fd18-4011-853a-ea273920df91 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:32.627857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3326ac8f-fd18-4011-853a-ea273920df91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.639333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 793/1611] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:32.641245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ba12f-d9c5-4515-96e4-21cea876b7fe None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.641641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ba12f-d9c5-4515-96e4-21cea876b7fe None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.642479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ba12f-d9c5-4515-96e4-21cea876b7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.689616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-5b8ba12f-d9c5-4515-96e4-21cea876b7fe None None] MFA Rules not processed for user `f697f7f3ddc142cba550e2385b26552f`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:32.691798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5b8ba12f-d9c5-4515-96e4-21cea876b7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.694480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 819/1612] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:32.695476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cced3-d55c-4b30-b1fb-c29c0487089d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.698096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-db1cced3-d55c-4b30-b1fb-c29c0487089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.783514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 794/1613] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:32.785786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cced3-d55c-4b30-b1fb-c29c0487089d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.789882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25f51f-9b15-4571-b8c5-78667da0b0aa None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.790247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25f51f-9b15-4571-b8c5-78667da0b0aa None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.790609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25f51f-9b15-4571-b8c5-78667da0b0aa None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.792181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 795/1614] 10.222.0.20 () {54 vars in 961 bytes} [Thu Jun 16 00:34:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:34:32.792926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25f51f-9b15-4571-b8c5-78667da0b0aa None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.794528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc0c2b-6cc3-49b6-a8fd-caa5f0a5d8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.795610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc0c2b-6cc3-49b6-a8fd-caa5f0a5d8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.796626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc0c2b-6cc3-49b6-a8fd-caa5f0a5d8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.797185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc0c2b-6cc3-49b6-a8fd-caa5f0a5d8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.806103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25f51f-9b15-4571-b8c5-78667da0b0aa None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.811078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970857e-ee8f-48a5-8542-938de2d4d6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.811340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc0c2b-6cc3-49b6-a8fd-caa5f0a5d8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.812205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970857e-ee8f-48a5-8542-938de2d4d6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.812558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970857e-ee8f-48a5-8542-938de2d4d6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.813325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970857e-ee8f-48a5-8542-938de2d4d6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.813450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc0c2b-6cc3-49b6-a8fd-caa5f0a5d8f4 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.827006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970857e-ee8f-48a5-8542-938de2d4d6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.827006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970857e-ee8f-48a5-8542-938de2d4d6b0 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.828698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 820/1615] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:32.829892 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc0c2b-6cc3-49b6-a8fd-caa5f0a5d8f4 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.839219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 796/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:32.840389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970857e-ee8f-48a5-8542-938de2d4d6b0 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.842068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc0c2b-6cc3-49b6-a8fd-caa5f0a5d8f4 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.848134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0a8c7-12fb-415b-b6a2-bd341323ff53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.849092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0a8c7-12fb-415b-b6a2-bd341323ff53 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.849399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0a8c7-12fb-415b-b6a2-bd341323ff53 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.851721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0a8c7-12fb-415b-b6a2-bd341323ff53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.855345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970857e-ee8f-48a5-8542-938de2d4d6b0 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.862987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbed1e7-9881-4fab-b1c4-28b82fdf275e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.864447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbed1e7-9881-4fab-b1c4-28b82fdf275e admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.864839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbed1e7-9881-4fab-b1c4-28b82fdf275e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.865131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbed1e7-9881-4fab-b1c4-28b82fdf275e admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.865888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbed1e7-9881-4fab-b1c4-28b82fdf275e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.868485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbed1e7-9881-4fab-b1c4-28b82fdf275e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.869523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0a8c7-12fb-415b-b6a2-bd341323ff53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.871526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0a8c7-12fb-415b-b6a2-bd341323ff53 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.890211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 821/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:32.891665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b734-8c1e-476e-8d86-bc9ec5095736 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.891961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b734-8c1e-476e-8d86-bc9ec5095736 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.892241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b734-8c1e-476e-8d86-bc9ec5095736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.900234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:32.916832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2897b734-8c1e-476e-8d86-bc9ec5095736 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:32.918024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2897b734-8c1e-476e-8d86-bc9ec5095736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.994543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 822/1618] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Jun 16 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:32.996101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444bfe-1054-442c-87b0-ac169d469769 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.996469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444bfe-1054-442c-87b0-ac169d469769 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.996872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444bfe-1054-442c-87b0-ac169d469769 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.998506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 823/1619] 10.222.0.20 () {54 vars in 961 bytes} [Thu Jun 16 00:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:34:33.005650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f0d4-8dc4-4e51-bc2a-d99b0c83ed24 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.010356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f0d4-8dc4-4e51-bc2a-d99b0c83ed24 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.010356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f0d4-8dc4-4e51-bc2a-d99b0c83ed24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.010559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 797/1620] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:32 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:33.010559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbed1e7-9881-4fab-b1c4-28b82fdf275e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.027984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-23d5f0d4-8dc4-4e51-bc2a-d99b0c83ed24 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:33.029209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-23d5f0d4-8dc4-4e51-bc2a-d99b0c83ed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:33.060854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:33.082643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbed1e7-9881-4fab-b1c4-28b82fdf275e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.088485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22bcbe-c329-49f4-90e9-e44508aaedfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.089205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22bcbe-c329-49f4-90e9-e44508aaedfc service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.089453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22bcbe-c329-49f4-90e9-e44508aaedfc service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.089735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22bcbe-c329-49f4-90e9-e44508aaedfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.102042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 824/1621] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Jun 16 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:33.102927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5f0d4-8dc4-4e51-bc2a-d99b0c83ed24 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.156303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22bcbe-c329-49f4-90e9-e44508aaedfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.157854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22bcbe-c329-49f4-90e9-e44508aaedfc service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.167544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5f0d4-8dc4-4e51-bc2a-d99b0c83ed24 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.168139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 798/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:33.173108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35682a98-ae8a-47f2-a03e-6278b4849e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.173798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35682a98-ae8a-47f2-a03e-6278b4849e45 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.174109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35682a98-ae8a-47f2-a03e-6278b4849e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.174386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35682a98-ae8a-47f2-a03e-6278b4849e45 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.174752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22bcbe-c329-49f4-90e9-e44508aaedfc service nova] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.175134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35682a98-ae8a-47f2-a03e-6278b4849e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.177315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35682a98-ae8a-47f2-a03e-6278b4849e45 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.190041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22bcbe-c329-49f4-90e9-e44508aaedfc service nova] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.196878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cea373-6b93-4ab0-b414-4fe5bc3d2525 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.197532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cea373-6b93-4ab0-b414-4fe5bc3d2525 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.197818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cea373-6b93-4ab0-b414-4fe5bc3d2525 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.198099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cea373-6b93-4ab0-b414-4fe5bc3d2525 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.254098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cea373-6b93-4ab0-b414-4fe5bc3d2525 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.255753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cea373-6b93-4ab0-b414-4fe5bc3d2525 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.265996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 799/1623] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:33.296325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 825/1624] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:33 2022] POST /identity/v3/users => generated 300 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:33.300103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cea373-6b93-4ab0-b414-4fe5bc3d2525 service nova] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.312431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cea373-6b93-4ab0-b414-4fe5bc3d2525 service nova] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.319387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a33742-43af-4858-8042-e812cf9d7b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.320118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a33742-43af-4858-8042-e812cf9d7b11 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.323117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a33742-43af-4858-8042-e812cf9d7b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.323117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a33742-43af-4858-8042-e812cf9d7b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.323117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a33742-43af-4858-8042-e812cf9d7b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.323416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a33742-43af-4858-8042-e812cf9d7b11 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.339840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 800/1625] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:33.340802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35682a98-ae8a-47f2-a03e-6278b4849e45 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.354579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35682a98-ae8a-47f2-a03e-6278b4849e45 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.361407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732593b-a7aa-49f6-b9b0-ca9893ad3d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.362628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732593b-a7aa-49f6-b9b0-ca9893ad3d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.363445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732593b-a7aa-49f6-b9b0-ca9893ad3d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.363976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732593b-a7aa-49f6-b9b0-ca9893ad3d7d admin admin] PATH_INFO: `/v3/projects/91961b3e9801463cb2efbb47b858547e/users/c060527e27134c2c9c87062dff50acfa/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.388666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732593b-a7aa-49f6-b9b0-ca9893ad3d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91961b3e9801463cb2efbb47b858547e, user_id=c060527e27134c2c9c87062dff50acfa, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.389693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732593b-a7aa-49f6-b9b0-ca9893ad3d7d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.413599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146fd9a-d8a2-4a7b-9b38-711325de0c72 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.413896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146fd9a-d8a2-4a7b-9b38-711325de0c72 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.414196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146fd9a-d8a2-4a7b-9b38-711325de0c72 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.416002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 801/1626] 10.222.0.20 () {54 vars in 912 bytes} [Thu Jun 16 00:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:34:33.421572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1975-42a3-4136-ab1a-333ee0df7a4d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.422209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1975-42a3-4136-ab1a-333ee0df7a4d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.422209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1975-42a3-4136-ab1a-333ee0df7a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.442949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-b19c1975-42a3-4136-ab1a-333ee0df7a4d None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:33.446624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b19c1975-42a3-4136-ab1a-333ee0df7a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:33.462548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 802/1627] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:33.481125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 826/1628] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:33 2022] PUT /identity/v3/projects/91961b3e9801463cb2efbb47b858547e/users/c060527e27134c2c9c87062dff50acfa/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:33.485377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c1975-42a3-4136-ab1a-333ee0df7a4d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.485834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732593b-a7aa-49f6-b9b0-ca9893ad3d7d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.568045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c1975-42a3-4136-ab1a-333ee0df7a4d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.569634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732593b-a7aa-49f6-b9b0-ca9893ad3d7d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.575040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40c3cc-d8a3-49ed-b077-d46e47df4738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.575655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40c3cc-d8a3-49ed-b077-d46e47df4738 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.575956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40c3cc-d8a3-49ed-b077-d46e47df4738 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.576276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40c3cc-d8a3-49ed-b077-d46e47df4738 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.580349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40c3cc-d8a3-49ed-b077-d46e47df4738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.589510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40c3cc-d8a3-49ed-b077-d46e47df4738 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.607562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 827/1629] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:33.616448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40c3cc-d8a3-49ed-b077-d46e47df4738 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.630090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40c3cc-d8a3-49ed-b077-d46e47df4738 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.635894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e3f44-5122-4ef4-b75d-2c6b28f682a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.637278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e3f44-5122-4ef4-b75d-2c6b28f682a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.638237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e3f44-5122-4ef4-b75d-2c6b28f682a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.638707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e3f44-5122-4ef4-b75d-2c6b28f682a7 admin admin] PATH_INFO: `/v3/projects/91961b3e9801463cb2efbb47b858547e/users/c060527e27134c2c9c87062dff50acfa/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.647555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e3f44-5122-4ef4-b75d-2c6b28f682a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91961b3e9801463cb2efbb47b858547e, user_id=c060527e27134c2c9c87062dff50acfa, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.666906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563af22-7b41-4d35-bdd0-f379dcde3942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.666906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563af22-7b41-4d35-bdd0-f379dcde3942 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.666906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563af22-7b41-4d35-bdd0-f379dcde3942 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.666906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563af22-7b41-4d35-bdd0-f379dcde3942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.667246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e3f44-5122-4ef4-b75d-2c6b28f682a7 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.667246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:33.692644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 828/1630] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:33 2022] PUT /identity/v3/projects/91961b3e9801463cb2efbb47b858547e/users/c060527e27134c2c9c87062dff50acfa/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:33.698350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67581896-f20f-4059-a7ca-f38481654a44 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.702014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67581896-f20f-4059-a7ca-f38481654a44 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.702014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67581896-f20f-4059-a7ca-f38481654a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.735947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-67581896-f20f-4059-a7ca-f38481654a44 None None] MFA Rules not processed for user `c060527e27134c2c9c87062dff50acfa`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:33.737781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-67581896-f20f-4059-a7ca-f38481654a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:33.740827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563af22-7b41-4d35-bdd0-f379dcde3942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.742489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563af22-7b41-4d35-bdd0-f379dcde3942 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.749938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:33.760113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 803/1631] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:33.763141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047adae6-28d9-4baa-9fc2-1e26ce74c439 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.763141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047adae6-28d9-4baa-9fc2-1e26ce74c439 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.763141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047adae6-28d9-4baa-9fc2-1e26ce74c439 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.764135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 804/1632] 10.222.0.20 () {54 vars in 912 bytes} [Thu Jun 16 00:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:34:33.771156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e0b97-af4f-4c1a-bace-8f9951fb7e35 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.771897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e0b97-af4f-4c1a-bace-8f9951fb7e35 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.772309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e0b97-af4f-4c1a-bace-8f9951fb7e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.798139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7e6e0b97-af4f-4c1a-bace-8f9951fb7e35 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:33.799385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7e6e0b97-af4f-4c1a-bace-8f9951fb7e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:33.843774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 829/1633] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:33.855413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67581896-f20f-4059-a7ca-f38481654a44 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.904474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 805/1634] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:33.927335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67581896-f20f-4059-a7ca-f38481654a44 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:33.931192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d8a6a-43ef-49b7-aff5-b699983ab564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:33.939621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8a6a-43ef-49b7-aff5-b699983ab564 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:33.939621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8a6a-43ef-49b7-aff5-b699983ab564 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:33.939621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8a6a-43ef-49b7-aff5-b699983ab564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:33.943389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d8a6a-43ef-49b7-aff5-b699983ab564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:33.945012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d8a6a-43ef-49b7-aff5-b699983ab564 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:33.956848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 830/1635] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:33.984614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e0b97-af4f-4c1a-bace-8f9951fb7e35 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.989699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d8a6a-43ef-49b7-aff5-b699983ab564 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:33.999954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d8a6a-43ef-49b7-aff5-b699983ab564 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:34.005640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74b614-eef2-4f3c-8737-1ce3ffe5fc39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:34.007798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74b614-eef2-4f3c-8737-1ce3ffe5fc39 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:34.009415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74b614-eef2-4f3c-8737-1ce3ffe5fc39 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:34.009711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74b614-eef2-4f3c-8737-1ce3ffe5fc39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:34.024292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74b614-eef2-4f3c-8737-1ce3ffe5fc39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:34.025933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74b614-eef2-4f3c-8737-1ce3ffe5fc39 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:34.037848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 831/1636] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:34.058039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e0b97-af4f-4c1a-bace-8f9951fb7e35 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:34.062919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:34.064412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:34.064732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:34.065017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] PATH_INFO: `/v3/users/025df89c03f044d0aa7fd0efea9afb53` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:34.066817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=025df89c03f044d0aa7fd0efea9afb53)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:34.070208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:34.093759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:34.204553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74b614-eef2-4f3c-8737-1ce3ffe5fc39 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:34.215196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74b614-eef2-4f3c-8737-1ce3ffe5fc39 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:34.219819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:34.220712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:34.221007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:34.221305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] PATH_INFO: `/v3/users/f697f7f3ddc142cba550e2385b26552f` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:34.223004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f697f7f3ddc142cba550e2385b26552f)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:34.224024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: WARNING py.warnings [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:34:34.224024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: warnings.warn(msg) Jun 16 00:34:34.224024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: Jun 16 00:34:34.225273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:34.363704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '025df89c03f044d0aa7fd0efea9afb53'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.370328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '025df89c03f044d0aa7fd0efea9afb53'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.371016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '025df89c03f044d0aa7fd0efea9afb53'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.371621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '025df89c03f044d0aa7fd0efea9afb53'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.406477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:34.498114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0dfc6b4c-4c04-4731-8a79-dce55842d53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '025df89c03f044d0aa7fd0efea9afb53'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.530091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 806/1637] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:33 2022] DELETE /identity/v3/users/025df89c03f044d0aa7fd0efea9afb53 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:34.537598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd737-8acf-4e8f-8d9b-8063b7fffd48 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:34.538186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd737-8acf-4e8f-8d9b-8063b7fffd48 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:34.538771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd737-8acf-4e8f-8d9b-8063b7fffd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:34.602859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-fabdd737-8acf-4e8f-8d9b-8063b7fffd48 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:34.607708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fabdd737-8acf-4e8f-8d9b-8063b7fffd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:34.624988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f697f7f3ddc142cba550e2385b26552f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.700638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f697f7f3ddc142cba550e2385b26552f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.780395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f697f7f3ddc142cba550e2385b26552f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.788318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f697f7f3ddc142cba550e2385b26552f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.793306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ecdee072-334f-4af9-9103-29e325f9267e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f697f7f3ddc142cba550e2385b26552f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:34.837417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 807/1638] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:34.873462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdd737-8acf-4e8f-8d9b-8063b7fffd48 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:34.875532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fabdd737-8acf-4e8f-8d9b-8063b7fffd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:34.896219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 832/1639] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:34 2022] DELETE /identity/v3/users/f697f7f3ddc142cba550e2385b26552f => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:34.902535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018755a-f997-4683-a0ec-19bc3d6ce2ee None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:34.902894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018755a-f997-4683-a0ec-19bc3d6ce2ee None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:34.903411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018755a-f997-4683-a0ec-19bc3d6ce2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:34.958049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-9018755a-f997-4683-a0ec-19bc3d6ce2ee None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:34.963187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9018755a-f997-4683-a0ec-19bc3d6ce2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:35.174620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdd737-8acf-4e8f-8d9b-8063b7fffd48 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:35.261344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:35.281764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 833/1640] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:35.321120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018755a-f997-4683-a0ec-19bc3d6ce2ee None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:35.329982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:35.335472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:35.335780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:35.339419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:35.341278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018755a-f997-4683-a0ec-19bc3d6ce2ee None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:35.360930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:35.363638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:35.363929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:35.367660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:35.373965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:35.376002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:35.394598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:35.394976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 808/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:35.395100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:35.423883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 834/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:37.150098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:37.152041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:37.220180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff3396-b26c-46d4-be65-9bd331705ddb service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:37.246416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:37.247527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:37.249090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:37.249513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] PATH_INFO: `/v3/projects/bfe70fb31ab14040b64268384c5458e9` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:37.252667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe70fb31ab14040b64268384c5458e9)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:37.253946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: WARNING py.warnings [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:34:37.253946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: warnings.warn(msg) Jun 16 00:34:37.253946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: Jun 16 00:34:37.255670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:37.291395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:37.293917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:37.325612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3afa8-b86f-4bc6-a26d-50e25ce53f7c service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:37.343267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:37.344167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:37.344444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:37.348951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] PATH_INFO: `/v3/projects/704cb48b399648e08be940e082da2c3e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:37.352925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=704cb48b399648e08be940e082da2c3e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:37.356850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:37.548333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:37.559200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704cb48b399648e08be940e082da2c3e', 'request_id': 'req-d0a323df-0d42-4f37-8062-a682e68f4a8b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.596148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe70fb31ab14040b64268384c5458e9', 'request_id': 'req-5532b587-f530-4f03-9498-b7171696ffc5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.707634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704cb48b399648e08be940e082da2c3e', 'request_id': 'req-d0a323df-0d42-4f37-8062-a682e68f4a8b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.721791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704cb48b399648e08be940e082da2c3e', 'request_id': 'req-d0a323df-0d42-4f37-8062-a682e68f4a8b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.761868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] The token cache is being invalidate because project 704cb48b399648e08be940e082da2c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:37.762277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.764231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0a323df-0d42-4f37-8062-a682e68f4a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.768095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 835/1643] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:37 2022] DELETE /identity/v3/projects/704cb48b399648e08be940e082da2c3e => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:37.774728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3ce1-0ad1-45c6-b8f4-aef23eb11969 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:37.775074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3ce1-0ad1-45c6-b8f4-aef23eb11969 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:37.776617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3ce1-0ad1-45c6-b8f4-aef23eb11969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:37.826116 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a51d3ce1-0ad1-45c6-b8f4-aef23eb11969 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:37.828278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a51d3ce1-0ad1-45c6-b8f4-aef23eb11969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:37.829037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe70fb31ab14040b64268384c5458e9', 'request_id': 'req-5532b587-f530-4f03-9498-b7171696ffc5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.840405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe70fb31ab14040b64268384c5458e9', 'request_id': 'req-5532b587-f530-4f03-9498-b7171696ffc5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.881984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] The token cache is being invalidate because project bfe70fb31ab14040b64268384c5458e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:37.882709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.885230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:37.888942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 809/1644] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:37 2022] DELETE /identity/v3/projects/bfe70fb31ab14040b64268384c5458e9 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:37.895062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:37.896689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:38.020993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 836/1645] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:38.027102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284672b3-a736-4e6f-8016-993888c8b169 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:38.027917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284672b3-a736-4e6f-8016-993888c8b169 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:38.028217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284672b3-a736-4e6f-8016-993888c8b169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:38.031165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532b587-f530-4f03-9498-b7171696ffc5 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:38.047507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:38.048509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:38.048929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:38.049228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] PATH_INFO: `/v3/users/c060527e27134c2c9c87062dff50acfa` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:38.051120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c060527e27134c2c9c87062dff50acfa)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:38.053153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:38.063008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-284672b3-a736-4e6f-8016-993888c8b169 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:38.065217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-284672b3-a736-4e6f-8016-993888c8b169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:38.082694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 837/1646] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:38.084357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284672b3-a736-4e6f-8016-993888c8b169 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:38.120003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284672b3-a736-4e6f-8016-993888c8b169 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:38.128554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:38.129469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:38.129890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:38.130324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:38.131559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:38.132015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:38.136438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:38.156505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 838/1647] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:38.157800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:38.171386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02307f89-e185-4256-9ba3-5d6b70359c9a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:38.178200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54e9f8-3f55-45c0-b1a2-c53636588e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:38.179496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54e9f8-3f55-45c0-b1a2-c53636588e42 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:38.179839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54e9f8-3f55-45c0-b1a2-c53636588e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:38.180155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54e9f8-3f55-45c0-b1a2-c53636588e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:38.180943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54e9f8-3f55-45c0-b1a2-c53636588e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:38.183564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54e9f8-3f55-45c0-b1a2-c53636588e42 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:38.346898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 839/1648] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:38 2022] POST /identity/v3/projects => generated 403 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:38.348426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54e9f8-3f55-45c0-b1a2-c53636588e42 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:38.349710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c060527e27134c2c9c87062dff50acfa'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:38.374716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c060527e27134c2c9c87062dff50acfa'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:38.393910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c060527e27134c2c9c87062dff50acfa'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:38.399037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c060527e27134c2c9c87062dff50acfa'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:38.411068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:38.486358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c060527e27134c2c9c87062dff50acfa'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:38.496898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54e9f8-3f55-45c0-b1a2-c53636588e42 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:38.508104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28283-f3bd-4f13-b7bf-feb844c2fa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:38.508630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28283-f3bd-4f13-b7bf-feb844c2fa0f admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:38.508920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28283-f3bd-4f13-b7bf-feb844c2fa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:38.509196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28283-f3bd-4f13-b7bf-feb844c2fa0f admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:38.509899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28283-f3bd-4f13-b7bf-feb844c2fa0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:38.511846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28283-f3bd-4f13-b7bf-feb844c2fa0f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:38.528079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 810/1649] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:37 2022] DELETE /identity/v3/users/c060527e27134c2c9c87062dff50acfa => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:38.675344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 840/1650] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:38 2022] POST /identity/v3/users => generated 302 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:38.679592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:38.681953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:38.829407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43d7fe-0384-49cc-b946-188c756d8fff admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:38.849855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4dac7-8742-431c-b51c-e0892bd9ae6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:38.850732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4dac7-8742-431c-b51c-e0892bd9ae6f admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:38.854008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4dac7-8742-431c-b51c-e0892bd9ae6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:38.854008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4dac7-8742-431c-b51c-e0892bd9ae6f admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:38.854008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4dac7-8742-431c-b51c-e0892bd9ae6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:38.854008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4dac7-8742-431c-b51c-e0892bd9ae6f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:38.875750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28283-f3bd-4f13-b7bf-feb844c2fa0f admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:38.885287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 811/1651] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:38.887670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28283-f3bd-4f13-b7bf-feb844c2fa0f admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:38.898792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b270b-2ba0-4e1b-af38-949116de1396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:38.899884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b270b-2ba0-4e1b-af38-949116de1396 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:38.900271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b270b-2ba0-4e1b-af38-949116de1396 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:38.901112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b270b-2ba0-4e1b-af38-949116de1396 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:38.901663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b270b-2ba0-4e1b-af38-949116de1396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:38.903505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b270b-2ba0-4e1b-af38-949116de1396 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:38.926817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4dac7-8742-431c-b51c-e0892bd9ae6f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:38.934952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 841/1652] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:38.942282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4dac7-8742-431c-b51c-e0892bd9ae6f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:38.954811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:38.956470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:38.957141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:38.957523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] PATH_INFO: `/v3/projects/db99dfcca0a54888a1298320b149323d/users/0c0133a208114aa0894aaa6b12ef07b6/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:38.986187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db99dfcca0a54888a1298320b149323d, user_id=0c0133a208114aa0894aaa6b12ef07b6, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:38.990442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:39.097986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb006e7f-e26a-48e4-bb39-e93c93833a64 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:39.098800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb006e7f-e26a-48e4-bb39-e93c93833a64 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:39.099187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb006e7f-e26a-48e4-bb39-e93c93833a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:39.110199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 812/1653] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:38 2022] PUT /identity/v3/projects/db99dfcca0a54888a1298320b149323d/users/0c0133a208114aa0894aaa6b12ef07b6/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:39.177273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-fb006e7f-e26a-48e4-bb39-e93c93833a64 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:39.203718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-fb006e7f-e26a-48e4-bb39-e93c93833a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:39.248642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:39.394240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 842/1654] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:39.452945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:39.452945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:39.684889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7f3a-e570-4970-a737-10fc72d4dc8a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:39.695313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65c7af-009b-468d-9886-2e6122c63d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:39.696020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65c7af-009b-468d-9886-2e6122c63d0f service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:39.697609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65c7af-009b-468d-9886-2e6122c63d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:39.697743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65c7af-009b-468d-9886-2e6122c63d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:39.725274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65c7af-009b-468d-9886-2e6122c63d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:39.727809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65c7af-009b-468d-9886-2e6122c63d0f service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:39.749995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 813/1655] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:39.881419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb006e7f-e26a-48e4-bb39-e93c93833a64 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:39.884786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-fb006e7f-e26a-48e4-bb39-e93c93833a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:39.961981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb006e7f-e26a-48e4-bb39-e93c93833a64 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:39.976115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:39.977339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:39.977826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:39.978300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] PATH_INFO: `/v3/projects/91961b3e9801463cb2efbb47b858547e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:39.981018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91961b3e9801463cb2efbb47b858547e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:39.987893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:40.181301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91961b3e9801463cb2efbb47b858547e', 'request_id': 'req-876d9d5c-2218-4e4a-be56-1377b053dc31'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.284254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91961b3e9801463cb2efbb47b858547e', 'request_id': 'req-876d9d5c-2218-4e4a-be56-1377b053dc31'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.293870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91961b3e9801463cb2efbb47b858547e', 'request_id': 'req-876d9d5c-2218-4e4a-be56-1377b053dc31'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.335544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] The token cache is being invalidate because project 91961b3e9801463cb2efbb47b858547e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:40.336772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.337819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-876d9d5c-2218-4e4a-be56-1377b053dc31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.340056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 843/1656] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:39 2022] DELETE /identity/v3/projects/91961b3e9801463cb2efbb47b858547e => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:40.344849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b150-3dc0-4afe-886c-d533b906419a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:40.345024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b150-3dc0-4afe-886c-d533b906419a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:40.345309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b150-3dc0-4afe-886c-d533b906419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:40.379299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e218b150-3dc0-4afe-886c-d533b906419a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:40.383062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e218b150-3dc0-4afe-886c-d533b906419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:40.572457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:40.626750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8a73a-8ae0-4bf4-8eb3-aae8cfa36d13 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:40.628106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8a73a-8ae0-4bf4-8eb3-aae8cfa36d13 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:40.628585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8a73a-8ae0-4bf4-8eb3-aae8cfa36d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:40.631192 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 814/1657] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:40.651702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-bab8a73a-8ae0-4bf4-8eb3-aae8cfa36d13 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:40.653168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-bab8a73a-8ae0-4bf4-8eb3-aae8cfa36d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:40.667225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b150-3dc0-4afe-886c-d533b906419a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:40.678569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 844/1658] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:40.679382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b150-3dc0-4afe-886c-d533b906419a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:40.685073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:40.686016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:40.686520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:40.686995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:40.687937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:40.688874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:40.691293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:40.709739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8a73a-8ae0-4bf4-8eb3-aae8cfa36d13 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:40.714495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 815/1659] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:40.720041 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8a73a-8ae0-4bf4-8eb3-aae8cfa36d13 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:40.726263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca009a-c642-4565-83b9-6cd4ac80f9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:40.727310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca009a-c642-4565-83b9-6cd4ac80f9e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:40.727799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca009a-c642-4565-83b9-6cd4ac80f9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:40.728270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca009a-c642-4565-83b9-6cd4ac80f9e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:40.729282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca009a-c642-4565-83b9-6cd4ac80f9e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:40.731598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca009a-c642-4565-83b9-6cd4ac80f9e6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:40.861478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:40.881451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 845/1660] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:40 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:40.885271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e0f0b-2a08-4071-83ca-fb3dcd0a3d14 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.043122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70146f4b-7962-4a97-bd76-8f9c9a80dcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.043829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70146f4b-7962-4a97-bd76-8f9c9a80dcd7 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.044120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70146f4b-7962-4a97-bd76-8f9c9a80dcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.044402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70146f4b-7962-4a97-bd76-8f9c9a80dcd7 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.045138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70146f4b-7962-4a97-bd76-8f9c9a80dcd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.047076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70146f4b-7962-4a97-bd76-8f9c9a80dcd7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.207276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca009a-c642-4565-83b9-6cd4ac80f9e6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.240510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 816/1661] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:40 2022] POST /identity/v3/users => generated 294 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:41.255244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca009a-c642-4565-83b9-6cd4ac80f9e6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.264359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e25c5-f091-42e3-8381-a8d789b7b035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.266595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e25c5-f091-42e3-8381-a8d789b7b035 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.266884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e25c5-f091-42e3-8381-a8d789b7b035 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.267209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e25c5-f091-42e3-8381-a8d789b7b035 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.268738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e25c5-f091-42e3-8381-a8d789b7b035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.271245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e25c5-f091-42e3-8381-a8d789b7b035 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.295994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70146f4b-7962-4a97-bd76-8f9c9a80dcd7 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.306342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 846/1662] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:41.310569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70146f4b-7962-4a97-bd76-8f9c9a80dcd7 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.318644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be20ad-131c-46b4-a7f6-40f77cf5a979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.319398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be20ad-131c-46b4-a7f6-40f77cf5a979 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.319965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be20ad-131c-46b4-a7f6-40f77cf5a979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.320250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be20ad-131c-46b4-a7f6-40f77cf5a979 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.321832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be20ad-131c-46b4-a7f6-40f77cf5a979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.323563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be20ad-131c-46b4-a7f6-40f77cf5a979 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.384316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e25c5-f091-42e3-8381-a8d789b7b035 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.402577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 817/1663] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:41.402767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e25c5-f091-42e3-8381-a8d789b7b035 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.420682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632bf1-e1a8-4d75-a10e-e8be674140a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.421602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632bf1-e1a8-4d75-a10e-e8be674140a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.421887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632bf1-e1a8-4d75-a10e-e8be674140a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.422166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632bf1-e1a8-4d75-a10e-e8be674140a1 admin admin] PATH_INFO: `/v3/projects/c1e92365d64d4b5a8464f33f0568ebf8/users/4db0f98267e7400ba8ad42e280dd857a/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.467814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632bf1-e1a8-4d75-a10e-e8be674140a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e92365d64d4b5a8464f33f0568ebf8, user_id=4db0f98267e7400ba8ad42e280dd857a, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.479683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632bf1-e1a8-4d75-a10e-e8be674140a1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.619662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec663-564e-47ff-9510-694d681c2fb4 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.620550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec663-564e-47ff-9510-694d681c2fb4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.620975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec663-564e-47ff-9510-694d681c2fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.623179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 847/1664] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:41 2022] PUT /identity/v3/projects/c1e92365d64d4b5a8464f33f0568ebf8/users/4db0f98267e7400ba8ad42e280dd857a/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:41.702698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f01ec663-564e-47ff-9510-694d681c2fb4 None None] MFA Rules not processed for user `4db0f98267e7400ba8ad42e280dd857a`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:41.706421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f01ec663-564e-47ff-9510-694d681c2fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:41.931822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 818/1665] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:41.943537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632bf1-e1a8-4d75-a10e-e8be674140a1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:42.017052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:42.090732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632bf1-e1a8-4d75-a10e-e8be674140a1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:42.096397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90756d1d-0e32-43ed-b149-8eabe36b5e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:42.097641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756d1d-0e32-43ed-b149-8eabe36b5e21 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:42.098381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756d1d-0e32-43ed-b149-8eabe36b5e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:42.098683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756d1d-0e32-43ed-b149-8eabe36b5e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.099465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90756d1d-0e32-43ed-b149-8eabe36b5e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:42.102487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90756d1d-0e32-43ed-b149-8eabe36b5e21 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:42.229764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 848/1666] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/projects => generated 387 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:42.237970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ec663-564e-47ff-9510-694d681c2fb4 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:42.254044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ec663-564e-47ff-9510-694d681c2fb4 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:42.361584 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:42.391310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bee2c-6450-4d7b-95ff-cd1cf0a97369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:42.392021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bee2c-6450-4d7b-95ff-cd1cf0a97369 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:42.392469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bee2c-6450-4d7b-95ff-cd1cf0a97369 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:42.392782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bee2c-6450-4d7b-95ff-cd1cf0a97369 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.394233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bee2c-6450-4d7b-95ff-cd1cf0a97369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:42.404636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bee2c-6450-4d7b-95ff-cd1cf0a97369 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:42.532803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 819/1667] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:42 2022] POST /identity/v3/users => generated 300 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:42.536501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90756d1d-0e32-43ed-b149-8eabe36b5e21 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:42.547831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90756d1d-0e32-43ed-b149-8eabe36b5e21 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:42.553861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9faf2-69be-48e8-b7e4-7cf17c57466d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:42.554938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9faf2-69be-48e8-b7e4-7cf17c57466d admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:42.555575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9faf2-69be-48e8-b7e4-7cf17c57466d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:42.556051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9faf2-69be-48e8-b7e4-7cf17c57466d admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.557032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9faf2-69be-48e8-b7e4-7cf17c57466d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:42.558889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9faf2-69be-48e8-b7e4-7cf17c57466d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:42.574452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 849/1668] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:42.583948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bee2c-6450-4d7b-95ff-cd1cf0a97369 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:42.603964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bee2c-6450-4d7b-95ff-cd1cf0a97369 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:42.619676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cceed8-ee64-4738-aec9-798cf3ebea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:42.620635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cceed8-ee64-4738-aec9-798cf3ebea4f admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:42.620905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cceed8-ee64-4738-aec9-798cf3ebea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:42.621185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cceed8-ee64-4738-aec9-798cf3ebea4f admin admin] PATH_INFO: `/v3/projects/5b25649820ba4ae68bc5a0e35c153282/users/46e18f4c78b14a6899b5af2375b5fdc8/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.658315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cceed8-ee64-4738-aec9-798cf3ebea4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b25649820ba4ae68bc5a0e35c153282, user_id=46e18f4c78b14a6899b5af2375b5fdc8, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:42.662478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cceed8-ee64-4738-aec9-798cf3ebea4f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:42.771676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9faf2-69be-48e8-b7e4-7cf17c57466d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:42.772882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 820/1669] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:42 2022] PUT /identity/v3/projects/5b25649820ba4ae68bc5a0e35c153282/users/46e18f4c78b14a6899b5af2375b5fdc8/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:42.786240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9faf2-69be-48e8-b7e4-7cf17c57466d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:42.932744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20883464-a1a4-4cc9-82e7-f2176ca48de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:42.933626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883464-a1a4-4cc9-82e7-f2176ca48de9 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:42.934060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883464-a1a4-4cc9-82e7-f2176ca48de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:42.934929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883464-a1a4-4cc9-82e7-f2176ca48de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.935836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20883464-a1a4-4cc9-82e7-f2176ca48de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:42.937720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20883464-a1a4-4cc9-82e7-f2176ca48de9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:42.964642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 850/1670] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:42.969831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cceed8-ee64-4738-aec9-798cf3ebea4f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:43.004142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cceed8-ee64-4738-aec9-798cf3ebea4f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:43.017979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:43.019025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:43.019309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:43.019644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] PATH_INFO: `/v3/projects/5b25649820ba4ae68bc5a0e35c153282/users/46e18f4c78b14a6899b5af2375b5fdc8/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:43.025396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b25649820ba4ae68bc5a0e35c153282, user_id=46e18f4c78b14a6899b5af2375b5fdc8, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:43.029999 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:43.066386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 821/1671] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:42 2022] PUT /identity/v3/projects/5b25649820ba4ae68bc5a0e35c153282/users/46e18f4c78b14a6899b5af2375b5fdc8/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:43.084394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b2b3-1818-4aa5-831c-e183981049b2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:43.084942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b2b3-1818-4aa5-831c-e183981049b2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:43.085399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b2b3-1818-4aa5-831c-e183981049b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:43.139005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1533b2b3-1818-4aa5-831c-e183981049b2 None None] MFA Rules not processed for user `46e18f4c78b14a6899b5af2375b5fdc8`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:43.140130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1533b2b3-1818-4aa5-831c-e183981049b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:43.298581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 851/1672] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:43.311018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:43.313483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:43.404785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96716e1f-767f-4bb9-801f-761840858783 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:43.410544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e7d0b-b256-454e-8d50-1bf81e5a1bb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:43.411311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e7d0b-b256-454e-8d50-1bf81e5a1bb4 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:43.427666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e7d0b-b256-454e-8d50-1bf81e5a1bb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:43.428465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e7d0b-b256-454e-8d50-1bf81e5a1bb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:43.456790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e7d0b-b256-454e-8d50-1bf81e5a1bb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:43.460905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e7d0b-b256-454e-8d50-1bf81e5a1bb4 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:43.508509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 822/1673] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:43.702362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b2b3-1818-4aa5-831c-e183981049b2 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:43.853073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:43.892263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b2b3-1818-4aa5-831c-e183981049b2 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:43.910744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:43.911433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:43.911732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:43.912014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] PATH_INFO: `/v3/users/4db0f98267e7400ba8ad42e280dd857a` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:43.914684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db0f98267e7400ba8ad42e280dd857a)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:43.917507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.217050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db0f98267e7400ba8ad42e280dd857a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.241842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db0f98267e7400ba8ad42e280dd857a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.263854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db0f98267e7400ba8ad42e280dd857a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.265109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db0f98267e7400ba8ad42e280dd857a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.265886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db0f98267e7400ba8ad42e280dd857a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.356957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 852/1674] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:43 2022] DELETE /identity/v3/users/4db0f98267e7400ba8ad42e280dd857a => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:44.361688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c39c41-5133-4f36-aece-82cca6963a45 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.362199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c39c41-5133-4f36-aece-82cca6963a45 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.362601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c39c41-5133-4f36-aece-82cca6963a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.390448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c4c39c41-5133-4f36-aece-82cca6963a45 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:44.391816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c4c39c41-5133-4f36-aece-82cca6963a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:44.561604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 823/1675] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:44.597443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:44.599309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:44.793256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a3e7-d0ce-440c-9198-81e686b79eb5 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.810307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.811230 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.820195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.821353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.836159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.838202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.852342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 853/1676] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:45.179043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799a3c8-9eaf-4b7d-9cad-20c27d49f548 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.179498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799a3c8-9eaf-4b7d-9cad-20c27d49f548 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.179940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799a3c8-9eaf-4b7d-9cad-20c27d49f548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.254105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9799a3c8-9eaf-4b7d-9cad-20c27d49f548 None None] MFA Rules not processed for user `0c0133a208114aa0894aaa6b12ef07b6`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:45.255706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9799a3c8-9eaf-4b7d-9cad-20c27d49f548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:45.340856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:45.398024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.399618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:45.424769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fadfe-3565-4927-aea3-1c672175cf09 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:45.449339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:45.452636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.455344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.456426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] PATH_INFO: `/v3/users/8166ce98a4d94d10aa6c28e84fa93fba` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.460348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8166ce98a4d94d10aa6c28e84fa93fba)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:45.464549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:45.484046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 824/1677] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:45.491008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799a3c8-9eaf-4b7d-9cad-20c27d49f548 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.499959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9799a3c8-9eaf-4b7d-9cad-20c27d49f548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:45.532813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799a3c8-9eaf-4b7d-9cad-20c27d49f548 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:45.551884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dbc06-d209-48b0-ab8a-f1cd22a01c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:45.553877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dbc06-d209-48b0-ab8a-f1cd22a01c4c admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.555543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dbc06-d209-48b0-ab8a-f1cd22a01c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.556103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dbc06-d209-48b0-ab8a-f1cd22a01c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.558873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dbc06-d209-48b0-ab8a-f1cd22a01c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:45.569559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dbc06-d209-48b0-ab8a-f1cd22a01c4c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:45.706052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 825/1678] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:45 2022] POST /identity/v3/projects => generated 403 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:45.707525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a23b3-d8ce-4220-b2a2-933871108e34 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.710316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a23b3-d8ce-4220-b2a2-933871108e34 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.710316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a23b3-d8ce-4220-b2a2-933871108e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.761948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ec6a23b3-d8ce-4220-b2a2-933871108e34 None None] MFA Rules not processed for user `052ae1db33014bb5915dc228f1b76d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:45.763536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ec6a23b3-d8ce-4220-b2a2-933871108e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:45.854805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:45.861035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8166ce98a4d94d10aa6c28e84fa93fba'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:45.902465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8166ce98a4d94d10aa6c28e84fa93fba'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:45.919708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8166ce98a4d94d10aa6c28e84fa93fba'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:45.921292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8166ce98a4d94d10aa6c28e84fa93fba'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:45.924876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8166ce98a4d94d10aa6c28e84fa93fba'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:45.989477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 826/1679] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:45.990672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a23b3-d8ce-4220-b2a2-933871108e34 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.992846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ec6a23b3-d8ce-4220-b2a2-933871108e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.055586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 854/1680] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:45 2022] DELETE /identity/v3/users/8166ce98a4d94d10aa6c28e84fa93fba => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:46.056705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:46.058234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.133426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a23b3-d8ce-4220-b2a2-933871108e34 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:46.213869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760d701-dde2-4c9f-9ffb-f4786a75b3bc admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:46.227636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23095f3-de71-4245-80e7-54bc5e4cb491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:46.228720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23095f3-de71-4245-80e7-54bc5e4cb491 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.229195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23095f3-de71-4245-80e7-54bc5e4cb491 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.229780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23095f3-de71-4245-80e7-54bc5e4cb491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.249640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f55e4a-989b-4593-9ea9-bdfc10541627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:46.250333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f55e4a-989b-4593-9ea9-bdfc10541627 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.250632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f55e4a-989b-4593-9ea9-bdfc10541627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.251472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f55e4a-989b-4593-9ea9-bdfc10541627 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.251577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f55e4a-989b-4593-9ea9-bdfc10541627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:46.254108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f55e4a-989b-4593-9ea9-bdfc10541627 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:46.336417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23095f3-de71-4245-80e7-54bc5e4cb491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:46.337112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23095f3-de71-4245-80e7-54bc5e4cb491 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:46.349271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 855/1681] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:46.352011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfa637-4ec7-4443-a5e1-9c51f6743d2b None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.352597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfa637-4ec7-4443-a5e1-9c51f6743d2b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.353093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfa637-4ec7-4443-a5e1-9c51f6743d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.358584 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 827/1682] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:45 2022] POST /identity/v3/users => generated 308 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:46.359890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2a1d0-9a9f-4ee8-ab8e-f70343ed1da1 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.365352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2a1d0-9a9f-4ee8-ab8e-f70343ed1da1 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.365352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2a1d0-9a9f-4ee8-ab8e-f70343ed1da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.395424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d5cfa637-4ec7-4443-a5e1-9c51f6743d2b None None] MFA Rules not processed for user `704b1cb48c7249d69ab69172d1a157b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:46.397237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d5cfa637-4ec7-4443-a5e1-9c51f6743d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.398427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e6e2a1d0-9a9f-4ee8-ab8e-f70343ed1da1 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:46.401883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e6e2a1d0-9a9f-4ee8-ab8e-f70343ed1da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.426764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 828/1683] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:46.427800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2a1d0-9a9f-4ee8-ab8e-f70343ed1da1 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:46.433383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e6e2a1d0-9a9f-4ee8-ab8e-f70343ed1da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.467681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2a1d0-9a9f-4ee8-ab8e-f70343ed1da1 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:46.481591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:46.482476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.482914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.483335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] PATH_INFO: `/v3/projects/c1e92365d64d4b5a8464f33f0568ebf8` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.487585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e92365d64d4b5a8464f33f0568ebf8)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:46.490416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:46.583657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 856/1684] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:46.583657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfa637-4ec7-4443-a5e1-9c51f6743d2b None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:46.601721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfa637-4ec7-4443-a5e1-9c51f6743d2b None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:46.610342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:46.611480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.612354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.612968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.613124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:46.614141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:46.618577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:46.636214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 857/1685] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:46.637069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:46.642338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.749645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e92365d64d4b5a8464f33f0568ebf8', 'request_id': 'req-df76bdf9-e1ed-439b-b1d5-50da4216da92'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:46.791278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04d4f-4106-43a0-a245-f3f43bfbd63e admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:46.891645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e92365d64d4b5a8464f33f0568ebf8', 'request_id': 'req-df76bdf9-e1ed-439b-b1d5-50da4216da92'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:46.899439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03625d8-5a6c-4dd8-b508-79c227d66409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:46.900471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03625d8-5a6c-4dd8-b508-79c227d66409 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.900786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03625d8-5a6c-4dd8-b508-79c227d66409 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.901162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03625d8-5a6c-4dd8-b508-79c227d66409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.905404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e92365d64d4b5a8464f33f0568ebf8', 'request_id': 'req-df76bdf9-e1ed-439b-b1d5-50da4216da92'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:46.938710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] The token cache is being invalidate because project c1e92365d64d4b5a8464f33f0568ebf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:46.939017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:46.943865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-df76bdf9-e1ed-439b-b1d5-50da4216da92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:46.949386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 829/1686] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:46 2022] DELETE /identity/v3/projects/c1e92365d64d4b5a8464f33f0568ebf8 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:46.950795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198c6e-741b-43e3-b497-acce328ab8b7 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.951087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198c6e-741b-43e3-b497-acce328ab8b7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.951373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198c6e-741b-43e3-b497-acce328ab8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:47.004864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-61198c6e-741b-43e3-b497-acce328ab8b7 None None] MFA Rules not processed for user `704b1cb48c7249d69ab69172d1a157b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:47.007564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-61198c6e-741b-43e3-b497-acce328ab8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.118245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03625d8-5a6c-4dd8-b508-79c227d66409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:47.119746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03625d8-5a6c-4dd8-b508-79c227d66409 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:47.147773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 858/1687] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:47.149824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03625d8-5a6c-4dd8-b508-79c227d66409 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:47.178499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03625d8-5a6c-4dd8-b508-79c227d66409 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:47.196448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:47.197426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:47.197721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:47.198157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] PATH_INFO: `/v3/projects/1c44371d0bec44599ffd0d7d54b504ad/users/6744bbcd367646cfbe78ca3bc4b30ae2/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:47.209000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 830/1688] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:47.209911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61198c6e-741b-43e3-b497-acce328ab8b7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:47.214146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-61198c6e-741b-43e3-b497-acce328ab8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.233479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c44371d0bec44599ffd0d7d54b504ad, user_id=6744bbcd367646cfbe78ca3bc4b30ae2, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:47.237301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:47.250861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61198c6e-741b-43e3-b497-acce328ab8b7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:47.265176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:47.266090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:47.266377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:47.266660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] PATH_INFO: `/v3/users/46e18f4c78b14a6899b5af2375b5fdc8` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:47.269429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e18f4c78b14a6899b5af2375b5fdc8)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:47.271250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:47.340957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 859/1689] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:47 2022] PUT /identity/v3/projects/1c44371d0bec44599ffd0d7d54b504ad/users/6744bbcd367646cfbe78ca3bc4b30ae2/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:47.340957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:47.340957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.465150 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0236969-fdaa-4519-a6a4-d554c0d77fb0 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:47.479699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:47.480764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:47.483209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:47.484635 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:47.488037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.567168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e18f4c78b14a6899b5af2375b5fdc8'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.583695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e18f4c78b14a6899b5af2375b5fdc8'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.586279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e18f4c78b14a6899b5af2375b5fdc8'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.587787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e18f4c78b14a6899b5af2375b5fdc8'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.628043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:47.629628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:47.652588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 860/1690] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:47.654757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:47.657974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.689885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e18f4c78b14a6899b5af2375b5fdc8'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.763018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:47.779599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 831/1691] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:47 2022] DELETE /identity/v3/users/46e18f4c78b14a6899b5af2375b5fdc8 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:47.870234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:47.872726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.900832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:47.926108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba5a22-7e4e-4056-8735-e151fac78122 service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.080022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.080881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.081174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.081459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.082208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.083839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.098312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db35ceb-a452-488a-929c-e350208d9631 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.112967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 861/1692] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:48.118445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.119268 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.119557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.119839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] PATH_INFO: `/v3/users/b6dee5e53bba4b3ba19b24cdfc3ae53c` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.122670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.125886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6dee5e53bba4b3ba19b24cdfc3ae53c)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.126778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.134811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.170266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f3a6c-86ef-4824-840c-559630bb44bb admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.182434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.185963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.185963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.185963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] PATH_INFO: `/v3/projects/1c44371d0bec44599ffd0d7d54b504ad/users/6744bbcd367646cfbe78ca3bc4b30ae2/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.191990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c44371d0bec44599ffd0d7d54b504ad, user_id=6744bbcd367646cfbe78ca3bc4b30ae2, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.194773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.220807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 862/1693] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:48 2022] PUT /identity/v3/projects/1c44371d0bec44599ffd0d7d54b504ad/users/6744bbcd367646cfbe78ca3bc4b30ae2/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:48.400619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.402525 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.431586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6dee5e53bba4b3ba19b24cdfc3ae53c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.456229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6dee5e53bba4b3ba19b24cdfc3ae53c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.460434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6dee5e53bba4b3ba19b24cdfc3ae53c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.462163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6dee5e53bba4b3ba19b24cdfc3ae53c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.533536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49502-3819-47c6-a9a3-238fbaf3fd85 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.540616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.541515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.541799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.542078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] PATH_INFO: `/v3/projects/a17db5f4c7b74c2f89cb295ce6cd8d80` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.543430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17db5f4c7b74c2f89cb295ce6cd8d80)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.545902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.551936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-432e8c48-45c6-4297-90c7-ee45ee43681b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6dee5e53bba4b3ba19b24cdfc3ae53c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.612572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 832/1694] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:47 2022] DELETE /identity/v3/users/b6dee5e53bba4b3ba19b24cdfc3ae53c => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:48.618062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e33e7-ec93-4a4e-984b-1505cb6934fe None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.618697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e33e7-ec93-4a4e-984b-1505cb6934fe None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.619218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e33e7-ec93-4a4e-984b-1505cb6934fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.654876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-293e33e7-ec93-4a4e-984b-1505cb6934fe None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:48.661010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-293e33e7-ec93-4a4e-984b-1505cb6934fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.790699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17db5f4c7b74c2f89cb295ce6cd8d80', 'request_id': 'req-776016ad-da5d-48dc-b0fa-02eb88d292d0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.865778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 833/1695] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:48.897101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e33e7-ec93-4a4e-984b-1505cb6934fe None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.897365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17db5f4c7b74c2f89cb295ce6cd8d80', 'request_id': 'req-776016ad-da5d-48dc-b0fa-02eb88d292d0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.898613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-293e33e7-ec93-4a4e-984b-1505cb6934fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.904675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17db5f4c7b74c2f89cb295ce6cd8d80', 'request_id': 'req-776016ad-da5d-48dc-b0fa-02eb88d292d0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.940989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] The token cache is being invalidate because project a17db5f4c7b74c2f89cb295ce6cd8d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:48.941291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.946136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-776016ad-da5d-48dc-b0fa-02eb88d292d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.957667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 863/1696] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:48 2022] DELETE /identity/v3/projects/a17db5f4c7b74c2f89cb295ce6cd8d80 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:48.964504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2aa8-343f-4cd1-9bca-5636021e32bf None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.964504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2aa8-343f-4cd1-9bca-5636021e32bf None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.964504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2aa8-343f-4cd1-9bca-5636021e32bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.038071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1e0c2aa8-343f-4cd1-9bca-5636021e32bf None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:49.039425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1e0c2aa8-343f-4cd1-9bca-5636021e32bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:49.125775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e33e7-ec93-4a4e-984b-1505cb6934fe None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.261382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 864/1697] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:49.273664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3f17-5e42-476a-bba5-dc28a1c205f2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.273971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3f17-5e42-476a-bba5-dc28a1c205f2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.274260 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3f17-5e42-476a-bba5-dc28a1c205f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.292253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.293720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.294033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.294329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.304867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:49.340422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f58d3f17-5e42-476a-bba5-dc28a1c205f2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:49.343558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f58d3f17-5e42-476a-bba5-dc28a1c205f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:49.346112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.348182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.364966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 865/1698] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:49.373913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d3f17-5e42-476a-bba5-dc28a1c205f2 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.389284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 834/1699] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:49.397034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d3f17-5e42-476a-bba5-dc28a1c205f2 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.403970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fff6e-d5fc-435c-9ee2-2283ccb17ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.405494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fff6e-d5fc-435c-9ee2-2283ccb17ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.405936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fff6e-d5fc-435c-9ee2-2283ccb17ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.406368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fff6e-d5fc-435c-9ee2-2283ccb17ec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.407261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fff6e-d5fc-435c-9ee2-2283ccb17ec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:49.408328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fff6e-d5fc-435c-9ee2-2283ccb17ec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.411568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fff6e-d5fc-435c-9ee2-2283ccb17ec4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.432840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.440272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 866/1700] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:49.449021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9764182-b8d4-443d-bc2f-813ba7379768 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.482506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125d02e-7fe6-4517-87d4-828453960007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.482506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d02e-7fe6-4517-87d4-828453960007 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.482506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d02e-7fe6-4517-87d4-828453960007 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.482506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d02e-7fe6-4517-87d4-828453960007 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.482876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125d02e-7fe6-4517-87d4-828453960007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.488668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125d02e-7fe6-4517-87d4-828453960007 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.574630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cab2f7-3708-48a2-b456-e96d78892b94 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.575030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cab2f7-3708-48a2-b456-e96d78892b94 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.575392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cab2f7-3708-48a2-b456-e96d78892b94 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.577602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 867/1701] 10.222.0.20 () {54 vars in 912 bytes} [Thu Jun 16 00:34:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:34:49.584797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827e823-c888-4072-b708-bb9e5b0ad8a0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.585185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827e823-c888-4072-b708-bb9e5b0ad8a0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.585562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827e823-c888-4072-b708-bb9e5b0ad8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.639572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 835/1702] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:49 2022] POST /identity/v3/projects => generated 357 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:49.641241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125d02e-7fe6-4517-87d4-828453960007 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.674034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e827e823-c888-4072-b708-bb9e5b0ad8a0 None None] MFA Rules not processed for user `96df1ee2cbc14bcc8a8a250fd8f349dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:49.680175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e827e823-c888-4072-b708-bb9e5b0ad8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:49.784992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:49.844883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125d02e-7fe6-4517-87d4-828453960007 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.851541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 868/1703] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:34:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:49.854745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.855443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.855734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.856944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.858091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.861611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.870420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:49.970497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827e823-c888-4072-b708-bb9e5b0ad8a0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.973416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 836/1704] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:49 2022] POST /identity/v3/users => generated 279 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:49.992778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827e823-c888-4072-b708-bb9e5b0ad8a0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:50.012904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5524-09c9-4d49-a89f-5781e9f0fef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:50.014488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5524-09c9-4d49-a89f-5781e9f0fef1 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.014779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5524-09c9-4d49-a89f-5781e9f0fef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.017854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:50.020186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:50.021699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5524-09c9-4d49-a89f-5781e9f0fef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.023947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd5524-09c9-4d49-a89f-5781e9f0fef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:50.025869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd5524-09c9-4d49-a89f-5781e9f0fef1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:50.043736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa5930-fb46-4ac5-b928-f8d548bc0dc4 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:50.048922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a08e15-cd48-4a9a-bcdc-da7ef14daeb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:50.050338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08e15-cd48-4a9a-bcdc-da7ef14daeb0 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.050918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08e15-cd48-4a9a-bcdc-da7ef14daeb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.051252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08e15-cd48-4a9a-bcdc-da7ef14daeb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.058021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 869/1705] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:50.059102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5524-09c9-4d49-a89f-5781e9f0fef1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:50.073055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a08e15-cd48-4a9a-bcdc-da7ef14daeb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:50.075801 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a08e15-cd48-4a9a-bcdc-da7ef14daeb0 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:50.083858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5524-09c9-4d49-a89f-5781e9f0fef1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:50.089119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 837/1706] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:50.093283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:50.094093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.094484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.094870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.095720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:50.097560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:50.126010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 870/1707] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:50.128168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a08e15-cd48-4a9a-bcdc-da7ef14daeb0 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:50.145621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a08e15-cd48-4a9a-bcdc-da7ef14daeb0 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:50.157329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb692fe1-2839-437a-90b3-81face3086f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:50.161492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb692fe1-2839-437a-90b3-81face3086f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.170738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb692fe1-2839-437a-90b3-81face3086f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.170738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb692fe1-2839-437a-90b3-81face3086f8 admin admin] PATH_INFO: `/v3/projects/1604f4a9e69d4667a255e02b0de2e538/users/08c1dc4c35d146b383af36fb271800d1/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.179909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:50.181911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:50.226467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba4631-4bd6-4163-b045-3311373d2be1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:50.233972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb692fe1-2839-437a-90b3-81face3086f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1604f4a9e69d4667a255e02b0de2e538, user_id=08c1dc4c35d146b383af36fb271800d1, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:50.237344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:50.238514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.238982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.239441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] PATH_INFO: `/v3/projects/5b25649820ba4ae68bc5a0e35c153282` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.242064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb692fe1-2839-437a-90b3-81face3086f8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:50.242978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b25649820ba4ae68bc5a0e35c153282)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:50.245158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:50.341308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 838/1708] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:50 2022] PUT /identity/v3/projects/1604f4a9e69d4667a255e02b0de2e538/users/08c1dc4c35d146b383af36fb271800d1/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:50.342734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c474-9a05-4a20-8478-89197da8abed None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.343032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c474-9a05-4a20-8478-89197da8abed None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.343313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c474-9a05-4a20-8478-89197da8abed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.384832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-eb90c474-9a05-4a20-8478-89197da8abed None None] MFA Rules not processed for user `31973e5a04c14cfc90ab377eac5f4194`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:50.386429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-eb90c474-9a05-4a20-8478-89197da8abed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:50.453923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b25649820ba4ae68bc5a0e35c153282', 'request_id': 'req-02d52180-05d7-4cdd-aa77-823f845cc622'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:50.484729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 839/1709] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:50.484729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1bc22-0b52-43e0-ac8f-7e63f0c89430 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.484729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1bc22-0b52-43e0-ac8f-7e63f0c89430 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.484729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1bc22-0b52-43e0-ac8f-7e63f0c89430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.521925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-77a1bc22-0b52-43e0-ac8f-7e63f0c89430 None None] MFA Rules not processed for user `08c1dc4c35d146b383af36fb271800d1`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:50.524420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-77a1bc22-0b52-43e0-ac8f-7e63f0c89430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:50.537591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b25649820ba4ae68bc5a0e35c153282', 'request_id': 'req-02d52180-05d7-4cdd-aa77-823f845cc622'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:50.546494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b25649820ba4ae68bc5a0e35c153282', 'request_id': 'req-02d52180-05d7-4cdd-aa77-823f845cc622'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:50.575416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] The token cache is being invalidate because project 5b25649820ba4ae68bc5a0e35c153282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:50.576603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:50.579064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:50.587261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 871/1710] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:50 2022] DELETE /identity/v3/projects/5b25649820ba4ae68bc5a0e35c153282 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:50.587261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:50.590435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:50.717256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 840/1711] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:50 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:50.721772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ff8ab-c0c2-4598-a677-657b359fe374 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.725149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ff8ab-c0c2-4598-a677-657b359fe374 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.728279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ff8ab-c0c2-4598-a677-657b359fe374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.765932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9c4ff8ab-c0c2-4598-a677-657b359fe374 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:50.767484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9c4ff8ab-c0c2-4598-a677-657b359fe374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:50.794247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:50.817779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d52180-05d7-4cdd-aa77-823f845cc622 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:50.834442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c921c-cd00-42ff-9c0d-1a4dc1dc4674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:50.835117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c921c-cd00-42ff-9c0d-1a4dc1dc4674 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.836991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c921c-cd00-42ff-9c0d-1a4dc1dc4674 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.836991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c921c-cd00-42ff-9c0d-1a4dc1dc4674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.844027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c41c921c-cd00-42ff-9c0d-1a4dc1dc4674 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:50.852578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 841/1712] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:50.852578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ff8ab-c0c2-4598-a677-657b359fe374 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:50.860074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9c4ff8ab-c0c2-4598-a677-657b359fe374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:50.899726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ff8ab-c0c2-4598-a677-657b359fe374 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:50.917102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffb938-d65a-4f88-bb69-84f77569ef29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:50.917788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffb938-d65a-4f88-bb69-84f77569ef29 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:50.918071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffb938-d65a-4f88-bb69-84f77569ef29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:50.918349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffb938-d65a-4f88-bb69-84f77569ef29 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:50.919056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffb938-d65a-4f88-bb69-84f77569ef29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:50.921002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffb938-d65a-4f88-bb69-84f77569ef29 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:51.051448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c921c-cd00-42ff-9c0d-1a4dc1dc4674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:51.053617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c921c-cd00-42ff-9c0d-1a4dc1dc4674 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:51.070140 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 872/1713] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:51.073960 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a32d-e88a-4304-85c1-4c89f2b27257 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.074401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a32d-e88a-4304-85c1-4c89f2b27257 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.074811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a32d-e88a-4304-85c1-4c89f2b27257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.089488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 842/1714] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:50 2022] POST /identity/v3/projects => generated 357 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:51.093377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffb938-d65a-4f88-bb69-84f77569ef29 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:51.106576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f120a32d-e88a-4304-85c1-4c89f2b27257 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:51.108296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f120a32d-e88a-4304-85c1-4c89f2b27257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:51.252282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 873/1715] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:51.256796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a32d-e88a-4304-85c1-4c89f2b27257 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:51.286463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a32d-e88a-4304-85c1-4c89f2b27257 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:51.298540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffb938-d65a-4f88-bb69-84f77569ef29 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:51.306706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:51.308175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.308468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.313190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.314273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:51.314590 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:51.322697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b7f08-0287-4b6f-9194-ba989308392b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:51.323400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b7f08-0287-4b6f-9194-ba989308392b admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.323685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b7f08-0287-4b6f-9194-ba989308392b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.323959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b7f08-0287-4b6f-9194-ba989308392b admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.326814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:51.332015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b7f08-0287-4b6f-9194-ba989308392b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:51.335070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b7f08-0287-4b6f-9194-ba989308392b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:51.368930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 874/1716] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:51.373057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:51.378312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:51.399470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:51.412470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd79e-253c-4418-a5fa-5d7659ea90c7 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:51.431416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:51.434092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.434092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.434313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] PATH_INFO: `/v3/projects/17623d8aa6b54f56a5e8e94d10c93bb7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.437293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17623d8aa6b54f56a5e8e94d10c93bb7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:51.444804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:51.479528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 843/1717] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:51 2022] POST /identity/v3/users => generated 285 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:51.480253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b7f08-0287-4b6f-9194-ba989308392b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:51.511581 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b7f08-0287-4b6f-9194-ba989308392b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:51.531926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598edbe-2253-4e39-b4e0-b5f77db79dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:51.534100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598edbe-2253-4e39-b4e0-b5f77db79dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.534100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598edbe-2253-4e39-b4e0-b5f77db79dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.534100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598edbe-2253-4e39-b4e0-b5f77db79dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.534100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598edbe-2253-4e39-b4e0-b5f77db79dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:51.538631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598edbe-2253-4e39-b4e0-b5f77db79dd7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:51.612269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:51.652859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17623d8aa6b54f56a5e8e94d10c93bb7', 'request_id': 'req-1657c31e-590d-455d-a6e8-794450eced40'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:51.717530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 844/1718] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:51 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:51.719779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679deb-ffae-42d3-9da4-fd9707eb1d9e None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.720217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679deb-ffae-42d3-9da4-fd9707eb1d9e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.720679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679deb-ffae-42d3-9da4-fd9707eb1d9e None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.722458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 845/1719] 10.222.0.20 () {54 vars in 960 bytes} [Thu Jun 16 00:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:34:51.723626 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa679deb-ffae-42d3-9da4-fd9707eb1d9e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:51.765773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17623d8aa6b54f56a5e8e94d10c93bb7', 'request_id': 'req-1657c31e-590d-455d-a6e8-794450eced40'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:51.773224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17623d8aa6b54f56a5e8e94d10c93bb7', 'request_id': 'req-1657c31e-590d-455d-a6e8-794450eced40'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:51.796836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] The token cache is being invalidate because project 17623d8aa6b54f56a5e8e94d10c93bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:51.797362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:51.800352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:51.804100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 875/1720] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:51 2022] DELETE /identity/v3/projects/17623d8aa6b54f56a5e8e94d10c93bb7 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:51.809693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:51.812388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:51.826254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa679deb-ffae-42d3-9da4-fd9707eb1d9e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:51.833759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676b5eb-dbe9-47ce-b1c2-e830312e82e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:51.834627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676b5eb-dbe9-47ce-b1c2-e830312e82e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.839325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676b5eb-dbe9-47ce-b1c2-e830312e82e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.839325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676b5eb-dbe9-47ce-b1c2-e830312e82e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.839325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676b5eb-dbe9-47ce-b1c2-e830312e82e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:51.839634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676b5eb-dbe9-47ce-b1c2-e830312e82e0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:51.856715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 846/1721] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:51.859246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7963b-6af3-45cf-a117-405ae07d6931 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.859675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7963b-6af3-45cf-a117-405ae07d6931 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.859976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7963b-6af3-45cf-a117-405ae07d6931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.897655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-aff7963b-6af3-45cf-a117-405ae07d6931 None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:51.900888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-aff7963b-6af3-45cf-a117-405ae07d6931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:51.934107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657c31e-590d-455d-a6e8-794450eced40 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:51.940713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806a3eb-a8d8-44b2-9ae2-aa51eae3046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:51.941722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806a3eb-a8d8-44b2-9ae2-aa51eae3046b admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:51.943497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806a3eb-a8d8-44b2-9ae2-aa51eae3046b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:51.943951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806a3eb-a8d8-44b2-9ae2-aa51eae3046b admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:51.944867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806a3eb-a8d8-44b2-9ae2-aa51eae3046b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:51.947394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806a3eb-a8d8-44b2-9ae2-aa51eae3046b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.036106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 876/1722] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:51 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:52.041157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464f356-c290-4807-aa39-91b75615eefe None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.041593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464f356-c290-4807-aa39-91b75615eefe None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.042159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464f356-c290-4807-aa39-91b75615eefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.058121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 847/1723] 10.222.0.20 () {58 vars in 1057 bytes} [Thu Jun 16 00:34:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:52.060469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7963b-6af3-45cf-a117-405ae07d6931 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.064691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b464f356-c290-4807-aa39-91b75615eefe None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:52.065369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-aff7963b-6af3-45cf-a117-405ae07d6931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:52.066699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b464f356-c290-4807-aa39-91b75615eefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:52.085766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 877/1724] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:52.088084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1079a3-5a27-4161-ba1a-e3ce58068e35 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.088628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1079a3-5a27-4161-ba1a-e3ce58068e35 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.089049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1079a3-5a27-4161-ba1a-e3ce58068e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.095305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7963b-6af3-45cf-a117-405ae07d6931 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.109473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e73934-4670-43c1-aa38-8a7e38540736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:52.110399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e73934-4670-43c1-aa38-8a7e38540736 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.110866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e73934-4670-43c1-aa38-8a7e38540736 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.111156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e73934-4670-43c1-aa38-8a7e38540736 admin admin] PATH_INFO: `/v3/projects/8510c11100914ad9b872c6d31a39f5ab/users/2bc26ecfd13547319c2ef23fc0aa8c22/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.114529 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6d1079a3-5a27-4161-ba1a-e3ce58068e35 None None] MFA Rules not processed for user `052ae1db33014bb5915dc228f1b76d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:52.116466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6d1079a3-5a27-4161-ba1a-e3ce58068e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:52.156339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e73934-4670-43c1-aa38-8a7e38540736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8510c11100914ad9b872c6d31a39f5ab, user_id=2bc26ecfd13547319c2ef23fc0aa8c22, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:52.163393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e73934-4670-43c1-aa38-8a7e38540736 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.214961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 878/1725] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:52.216265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1079a3-5a27-4161-ba1a-e3ce58068e35 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.235141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1079a3-5a27-4161-ba1a-e3ce58068e35 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.246180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4448a05-7ec1-493e-9b00-aa4a90ff4f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:52.246730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4448a05-7ec1-493e-9b00-aa4a90ff4f2a admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.247004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4448a05-7ec1-493e-9b00-aa4a90ff4f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.247464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4448a05-7ec1-493e-9b00-aa4a90ff4f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.248366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4448a05-7ec1-493e-9b00-aa4a90ff4f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:52.250356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4448a05-7ec1-493e-9b00-aa4a90ff4f2a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.261244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 848/1726] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:52 2022] PUT /identity/v3/projects/8510c11100914ad9b872c6d31a39f5ab/users/2bc26ecfd13547319c2ef23fc0aa8c22/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:52.262102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e73934-4670-43c1-aa38-8a7e38540736 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.263454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 879/1727] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:52.268951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97871df9-67d3-45f4-a084-92f49608a4fd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.269407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97871df9-67d3-45f4-a084-92f49608a4fd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.269846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97871df9-67d3-45f4-a084-92f49608a4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.293660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-97871df9-67d3-45f4-a084-92f49608a4fd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:52.294875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-97871df9-67d3-45f4-a084-92f49608a4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:52.437963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 880/1728] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:52.454524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97871df9-67d3-45f4-a084-92f49608a4fd None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.465979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e73934-4670-43c1-aa38-8a7e38540736 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.478875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97871df9-67d3-45f4-a084-92f49608a4fd None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.480254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:52.480970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.481263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.481554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.494521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe946b0c-1553-4d0c-bce6-954e83f07108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:52.495456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe946b0c-1553-4d0c-bce6-954e83f07108 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.495888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe946b0c-1553-4d0c-bce6-954e83f07108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.496316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe946b0c-1553-4d0c-bce6-954e83f07108 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.497279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe946b0c-1553-4d0c-bce6-954e83f07108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:52.498350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:52.505013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe946b0c-1553-4d0c-bce6-954e83f07108 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.521440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 881/1729] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:52.522620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe946b0c-1553-4d0c-bce6-954e83f07108 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.538365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:52.538365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.538901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe946b0c-1553-4d0c-bce6-954e83f07108 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.554209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73afcd9-fcf4-491b-bd45-f0dab468f205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:52.555017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73afcd9-fcf4-491b-bd45-f0dab468f205 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.555241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73afcd9-fcf4-491b-bd45-f0dab468f205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.555550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73afcd9-fcf4-491b-bd45-f0dab468f205 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.557387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73afcd9-fcf4-491b-bd45-f0dab468f205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:52.559991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73afcd9-fcf4-491b-bd45-f0dab468f205 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.561161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 849/1730] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:52.562204 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.580174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f67aba-9244-47db-8c5b-f638be11f791 service placement] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.598293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:52.599156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.599648 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.600075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.609506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:52.610048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:52.614126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.617581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 882/1731] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:52.618316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73afcd9-fcf4-491b-bd45-f0dab468f205 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.630569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 850/1732] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:52.631681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.646334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73afcd9-fcf4-491b-bd45-f0dab468f205 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.662916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf6b23-6d11-4077-bce4-7ffcec4317cc admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.663283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767187e0-9c25-42b5-b12b-1d71eeb763d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:52.664692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767187e0-9c25-42b5-b12b-1d71eeb763d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.666179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767187e0-9c25-42b5-b12b-1d71eeb763d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.666485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767187e0-9c25-42b5-b12b-1d71eeb763d5 admin admin] PATH_INFO: `/v3/projects/8510c11100914ad9b872c6d31a39f5ab/users/2bc26ecfd13547319c2ef23fc0aa8c22/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.671696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd80630-3f01-417d-8990-5093b446eb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:52.673045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd80630-3f01-417d-8990-5093b446eb5a admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.673482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd80630-3f01-417d-8990-5093b446eb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.675044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767187e0-9c25-42b5-b12b-1d71eeb763d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8510c11100914ad9b872c6d31a39f5ab, user_id=2bc26ecfd13547319c2ef23fc0aa8c22, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:52.679282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd80630-3f01-417d-8990-5093b446eb5a admin admin] PATH_INFO: `/v3/projects/513f80928bb34def978c26254b5152f7/users/dceab1b1494b43d0a936e9d0237e9da4/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.683162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767187e0-9c25-42b5-b12b-1d71eeb763d5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.756373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd80630-3f01-417d-8990-5093b446eb5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=513f80928bb34def978c26254b5152f7, user_id=dceab1b1494b43d0a936e9d0237e9da4, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:52.767787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 883/1733] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:52 2022] PUT /identity/v3/projects/8510c11100914ad9b872c6d31a39f5ab/users/2bc26ecfd13547319c2ef23fc0aa8c22/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:52.770741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767187e0-9c25-42b5-b12b-1d71eeb763d5 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:52.771717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd80630-3f01-417d-8990-5093b446eb5a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:52.896590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 851/1734] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:52 2022] PUT /identity/v3/projects/513f80928bb34def978c26254b5152f7/users/dceab1b1494b43d0a936e9d0237e9da4/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:52.899681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65293a4c-8a70-48c4-8746-120c552cbb8f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:52.899970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65293a4c-8a70-48c4-8746-120c552cbb8f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:52.901245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767187e0-9c25-42b5-b12b-1d71eeb763d5 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:52.902100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65293a4c-8a70-48c4-8746-120c552cbb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:52.918590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:52.998121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-65293a4c-8a70-48c4-8746-120c552cbb8f None None] MFA Rules not processed for user `2bc26ecfd13547319c2ef23fc0aa8c22`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:52.999656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-65293a4c-8a70-48c4-8746-120c552cbb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:53.093726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:53.097163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:53.097163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:53.097163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:53.097163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:53.097745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:53.209942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 852/1735] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:53.212794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24bbb-ac09-4a67-b616-ad95bab679b8 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:53.213091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24bbb-ac09-4a67-b616-ad95bab679b8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:53.214854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24bbb-ac09-4a67-b616-ad95bab679b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:53.272121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 884/1736] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:52 2022] POST /identity/v3/projects => generated 373 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:53.276453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:53.284412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:53.297049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4eb24bbb-ac09-4a67-b616-ad95bab679b8 None None] MFA Rules not processed for user `dceab1b1494b43d0a936e9d0237e9da4`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:53.304062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4eb24bbb-ac09-4a67-b616-ad95bab679b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:53.519735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6664f2e-a9b9-43e6-9ff9-c9931a511bc5 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:53.536423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43933e13-2369-4ac0-85a1-091d85c714f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:53.539015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43933e13-2369-4ac0-85a1-091d85c714f8 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:53.539313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43933e13-2369-4ac0-85a1-091d85c714f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:53.539602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43933e13-2369-4ac0-85a1-091d85c714f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:53.597953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 853/1737] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:53.599114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb24bbb-ac09-4a67-b616-ad95bab679b8 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:53.718989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:53.805484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43933e13-2369-4ac0-85a1-091d85c714f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:53.806982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43933e13-2369-4ac0-85a1-091d85c714f8 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:53.863404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb24bbb-ac09-4a67-b616-ad95bab679b8 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:53.865725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 885/1738] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:53 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:53.866707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43933e13-2369-4ac0-85a1-091d85c714f8 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:53.873887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4df84-b68d-42ce-9d76-4b6e3b771e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:53.876876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4df84-b68d-42ce-9d76-4b6e3b771e23 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:53.877173 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4df84-b68d-42ce-9d76-4b6e3b771e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:53.877492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4df84-b68d-42ce-9d76-4b6e3b771e23 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:53.878215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4df84-b68d-42ce-9d76-4b6e3b771e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:53.880200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4df84-b68d-42ce-9d76-4b6e3b771e23 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:53.903586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43933e13-2369-4ac0-85a1-091d85c714f8 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:53.923567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ba51c-f7e7-486a-8552-624777198a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:53.924289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ba51c-f7e7-486a-8552-624777198a6b admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:53.924624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ba51c-f7e7-486a-8552-624777198a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:53.933640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ba51c-f7e7-486a-8552-624777198a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:53.933640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ba51c-f7e7-486a-8552-624777198a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:53.933640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ba51c-f7e7-486a-8552-624777198a6b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:54.058342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 854/1739] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:53 2022] POST /identity/v3/users => generated 287 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:54.059755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154194d6-99fc-42a6-9f6f-be46b7f3407f None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.060026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154194d6-99fc-42a6-9f6f-be46b7f3407f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.060309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154194d6-99fc-42a6-9f6f-be46b7f3407f None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.064191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 855/1740] 10.222.0.20 () {54 vars in 912 bytes} [Thu Jun 16 00:34:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:34:54.065489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4aa86-cdc9-4435-b86f-7a03f4e40726 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.065964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4aa86-cdc9-4435-b86f-7a03f4e40726 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.067962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4aa86-cdc9-4435-b86f-7a03f4e40726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.122307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 886/1741] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:53 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:54.124047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ba51c-f7e7-486a-8552-624777198a6b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:54.148055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-afe4aa86-cdc9-4435-b86f-7a03f4e40726 None None] MFA Rules not processed for user `6744bbcd367646cfbe78ca3bc4b30ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:54.150614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-afe4aa86-cdc9-4435-b86f-7a03f4e40726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:54.311307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ba51c-f7e7-486a-8552-624777198a6b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:54.323599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 856/1742] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:54 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:54.325329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3503c-9682-4cec-b6bc-60c4bcb62d5b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.325992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3503c-9682-4cec-b6bc-60c4bcb62d5b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.326737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3503c-9682-4cec-b6bc-60c4bcb62d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.329163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519233aa-792b-437d-b571-9f9ea0fc0985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:54.329925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519233aa-792b-437d-b571-9f9ea0fc0985 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.330218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519233aa-792b-437d-b571-9f9ea0fc0985 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.335820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519233aa-792b-437d-b571-9f9ea0fc0985 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.336666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519233aa-792b-437d-b571-9f9ea0fc0985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:54.338604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519233aa-792b-437d-b571-9f9ea0fc0985 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:54.361409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 887/1743] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:54.362833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519233aa-792b-437d-b571-9f9ea0fc0985 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:54.387035 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519233aa-792b-437d-b571-9f9ea0fc0985 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:54.402087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58d833-d084-4153-a8a5-111df7797bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:54.402795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58d833-d084-4153-a8a5-111df7797bca admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.403083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58d833-d084-4153-a8a5-111df7797bca admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.403367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58d833-d084-4153-a8a5-111df7797bca admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.404081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58d833-d084-4153-a8a5-111df7797bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:54.410536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58d833-d084-4153-a8a5-111df7797bca admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:54.537773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-16f3503c-9682-4cec-b6bc-60c4bcb62d5b None None] MFA Rules not processed for user `96df1ee2cbc14bcc8a8a250fd8f349dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:54.539283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-16f3503c-9682-4cec-b6bc-60c4bcb62d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:54.608792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 888/1744] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:54 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:54.609736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58d833-d084-4153-a8a5-111df7797bca admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:54.631486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 857/1745] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:34:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:54.633021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae55c07-90bd-41f7-9fa0-60d6bce0b096 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.633315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae55c07-90bd-41f7-9fa0-60d6bce0b096 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.633598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae55c07-90bd-41f7-9fa0-60d6bce0b096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.668308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-0ae55c07-90bd-41f7-9fa0-60d6bce0b096 None None] MFA Rules not processed for user `c9dcfb252f0d44a280b3448094883e13`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:54.669617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-0ae55c07-90bd-41f7-9fa0-60d6bce0b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:54.760669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58d833-d084-4153-a8a5-111df7797bca admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:54.773536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:54.773743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.774343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.776740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 858/1746] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:54.777573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.778876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae55c07-90bd-41f7-9fa0-60d6bce0b096 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:54.783122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:54.810888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae55c07-90bd-41f7-9fa0-60d6bce0b096 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:54.822053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5046de-9660-4988-9772-dbba965cdf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:54.822762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5046de-9660-4988-9772-dbba965cdf07 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.824239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5046de-9660-4988-9772-dbba965cdf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.824239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5046de-9660-4988-9772-dbba965cdf07 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.824239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5046de-9660-4988-9772-dbba965cdf07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:54.825794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5046de-9660-4988-9772-dbba965cdf07 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:54.839332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 859/1747] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:54.840353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5046de-9660-4988-9772-dbba965cdf07 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:54.856730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5046de-9660-4988-9772-dbba965cdf07 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:54.865630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13e023-9252-41d4-afbd-e895d0ec11f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:54.866765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13e023-9252-41d4-afbd-e895d0ec11f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.869013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13e023-9252-41d4-afbd-e895d0ec11f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.869305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13e023-9252-41d4-afbd-e895d0ec11f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.870062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13e023-9252-41d4-afbd-e895d0ec11f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:54.872816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13e023-9252-41d4-afbd-e895d0ec11f1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:54.878348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:54.925383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:54.927905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 860/1748] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:54.930482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8c996-bb63-484a-85dd-b76d1f7bfcbe None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:54.930801 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8c996-bb63-484a-85dd-b76d1f7bfcbe None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:54.931099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8c996-bb63-484a-85dd-b76d1f7bfcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:54.938645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:54.963752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 889/1749] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:54 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:54.970262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:54.975545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:54.989887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788135-9d50-4566-8daa-dee849b84884 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:55.007815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0ec2e-b604-459c-827e-a216f2d2d6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:55.009348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0ec2e-b604-459c-827e-a216f2d2d6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:55.009585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0ec2e-b604-459c-827e-a216f2d2d6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:55.009941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0ec2e-b604-459c-827e-a216f2d2d6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:55.030404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0ec2e-b604-459c-827e-a216f2d2d6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:55.030404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0ec2e-b604-459c-827e-a216f2d2d6f5 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:55.039983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-bdd8c996-bb63-484a-85dd-b76d1f7bfcbe None None] MFA Rules not processed for user `c9dcfb252f0d44a280b3448094883e13`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:55.044383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-bdd8c996-bb63-484a-85dd-b76d1f7bfcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:55.046494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 890/1750] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:55.047388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0ec2e-b604-459c-827e-a216f2d2d6f5 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:55.087828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 861/1751] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:55.088832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8c996-bb63-484a-85dd-b76d1f7bfcbe None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:55.089125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0ec2e-b604-459c-827e-a216f2d2d6f5 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:55.095305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd143e1-2a47-478f-ad36-13739814bdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:55.096304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd143e1-2a47-478f-ad36-13739814bdd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:55.096627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd143e1-2a47-478f-ad36-13739814bdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:55.096968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd143e1-2a47-478f-ad36-13739814bdd5 admin admin] PATH_INFO: `/v3/projects/0bfae81d2de24f5bb456de555da2e229/users/542b5913990f40c5adc90d4685f559ec/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:55.138014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8c996-bb63-484a-85dd-b76d1f7bfcbe None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:55.140630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd143e1-2a47-478f-ad36-13739814bdd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bfae81d2de24f5bb456de555da2e229, user_id=542b5913990f40c5adc90d4685f559ec, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:55.144570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd143e1-2a47-478f-ad36-13739814bdd5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:55.156459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:55.157209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:55.157478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:55.157758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:55.158690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:55.160311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:55.186449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 862/1752] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:55.191597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:55.203689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:55.262354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 891/1753] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:55 2022] PUT /identity/v3/projects/0bfae81d2de24f5bb456de555da2e229/users/542b5913990f40c5adc90d4685f559ec/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:55.263414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd143e1-2a47-478f-ad36-13739814bdd5 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:55.414281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c08b1-3a59-42ba-a7ad-238d3486ac50 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:55.469965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd143e1-2a47-478f-ad36-13739814bdd5 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:55.484817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4b6ba-0161-409b-a8fb-6c987d8f3281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:55.484817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4b6ba-0161-409b-a8fb-6c987d8f3281 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:55.484817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4b6ba-0161-409b-a8fb-6c987d8f3281 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:55.484817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4b6ba-0161-409b-a8fb-6c987d8f3281 admin admin] PATH_INFO: `/v3/projects/5633dda8fca54f9b84488004f2d2b050/users/2e5a72e163bd4a818ad0502e7cdc8905/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:55.529504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4b6ba-0161-409b-a8fb-6c987d8f3281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5633dda8fca54f9b84488004f2d2b050, user_id=2e5a72e163bd4a818ad0502e7cdc8905, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:55.532114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4b6ba-0161-409b-a8fb-6c987d8f3281 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:55.537935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7995d-b849-4623-b0b2-9ae78cbaf021 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:55.538840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7995d-b849-4623-b0b2-9ae78cbaf021 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:55.539309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7995d-b849-4623-b0b2-9ae78cbaf021 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:55.539968 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7995d-b849-4623-b0b2-9ae78cbaf021 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:55.648568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 892/1754] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:55 2022] PUT /identity/v3/projects/5633dda8fca54f9b84488004f2d2b050/users/2e5a72e163bd4a818ad0502e7cdc8905/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:55.658355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e5ef6-13b8-4b02-baaa-fa7f14bb7601 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:55.658355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e5ef6-13b8-4b02-baaa-fa7f14bb7601 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:55.658355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e5ef6-13b8-4b02-baaa-fa7f14bb7601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:55.728985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7995d-b849-4623-b0b2-9ae78cbaf021 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:55.741823 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7995d-b849-4623-b0b2-9ae78cbaf021 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:55.785738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 863/1755] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:55.785738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba72cc-1f3a-48cb-8f30-7ff9460216e2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:55.785738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba72cc-1f3a-48cb-8f30-7ff9460216e2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:55.785738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba72cc-1f3a-48cb-8f30-7ff9460216e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:55.789941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4d5e5ef6-13b8-4b02-baaa-fa7f14bb7601 None None] MFA Rules not processed for user `542b5913990f40c5adc90d4685f559ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:55.791084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4d5e5ef6-13b8-4b02-baaa-fa7f14bb7601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:55.844888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-79ba72cc-1f3a-48cb-8f30-7ff9460216e2 None None] MFA Rules not processed for user `2e5a72e163bd4a818ad0502e7cdc8905`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:55.855748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-79ba72cc-1f3a-48cb-8f30-7ff9460216e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:55.930934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 893/1756] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:55.933567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e5ef6-13b8-4b02-baaa-fa7f14bb7601 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:55.994176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 864/1757] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:55.998264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba72cc-1f3a-48cb-8f30-7ff9460216e2 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:56.078047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e5ef6-13b8-4b02-baaa-fa7f14bb7601 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:56.088461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00b3d1-109a-4b89-ba54-46de2e4f2c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:56.089957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00b3d1-109a-4b89-ba54-46de2e4f2c6a admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:56.090262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00b3d1-109a-4b89-ba54-46de2e4f2c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:56.090546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00b3d1-109a-4b89-ba54-46de2e4f2c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:56.091663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00b3d1-109a-4b89-ba54-46de2e4f2c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:56.098060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00b3d1-109a-4b89-ba54-46de2e4f2c6a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:56.161138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba72cc-1f3a-48cb-8f30-7ff9460216e2 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:56.171778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d96232-c796-4054-b6ff-f304d6eb8c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:56.172475 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d96232-c796-4054-b6ff-f304d6eb8c6a admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:56.172806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d96232-c796-4054-b6ff-f304d6eb8c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:56.173090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d96232-c796-4054-b6ff-f304d6eb8c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:56.173800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d96232-c796-4054-b6ff-f304d6eb8c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:56.175740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d96232-c796-4054-b6ff-f304d6eb8c6a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:56.220893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 894/1758] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:55 2022] POST /identity/v3/projects => generated 371 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:56.224494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00b3d1-109a-4b89-ba54-46de2e4f2c6a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:56.236916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:56.325520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 865/1759] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:55 2022] POST /identity/v3/projects => generated 387 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:56.326578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d96232-c796-4054-b6ff-f304d6eb8c6a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:56.411336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00b3d1-109a-4b89-ba54-46de2e4f2c6a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:56.504907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f708e2-a8be-45b1-861d-538b141ddcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:56.505612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f708e2-a8be-45b1-861d-538b141ddcc9 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:56.505900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f708e2-a8be-45b1-861d-538b141ddcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:56.506321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f708e2-a8be-45b1-861d-538b141ddcc9 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:56.508059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f708e2-a8be-45b1-861d-538b141ddcc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:56.513810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f708e2-a8be-45b1-861d-538b141ddcc9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:56.547437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d96232-c796-4054-b6ff-f304d6eb8c6a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:56.554702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27affb6-8abd-4fb1-84c6-4c36954ee4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:56.555440 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27affb6-8abd-4fb1-84c6-4c36954ee4ed admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:56.555897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27affb6-8abd-4fb1-84c6-4c36954ee4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:56.556715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27affb6-8abd-4fb1-84c6-4c36954ee4ed admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:56.557657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27affb6-8abd-4fb1-84c6-4c36954ee4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:56.564881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27affb6-8abd-4fb1-84c6-4c36954ee4ed admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:56.657648 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:56.693583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 895/1760] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:56 2022] POST /identity/v3/users => generated 292 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:56.697354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f708e2-a8be-45b1-861d-538b141ddcc9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:56.698132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 866/1761] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:56 2022] POST /identity/v3/users => generated 300 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:56.699668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27affb6-8abd-4fb1-84c6-4c36954ee4ed admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:56.713148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27affb6-8abd-4fb1-84c6-4c36954ee4ed admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:56.720570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ff475-146e-4b38-9519-3080da6ca637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:56.722482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ff475-146e-4b38-9519-3080da6ca637 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:56.722776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ff475-146e-4b38-9519-3080da6ca637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:56.723189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f708e2-a8be-45b1-861d-538b141ddcc9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:56.723296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ff475-146e-4b38-9519-3080da6ca637 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:56.724732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ff475-146e-4b38-9519-3080da6ca637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:56.725866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ff475-146e-4b38-9519-3080da6ca637 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:56.734134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f297cc8-f46e-49d1-80e6-0a05ee048bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:56.735174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f297cc8-f46e-49d1-80e6-0a05ee048bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:56.735283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f297cc8-f46e-49d1-80e6-0a05ee048bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:56.735533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f297cc8-f46e-49d1-80e6-0a05ee048bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:56.736445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f297cc8-f46e-49d1-80e6-0a05ee048bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:56.738512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f297cc8-f46e-49d1-80e6-0a05ee048bf6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:56.747413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 867/1762] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:56.748562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ff475-146e-4b38-9519-3080da6ca637 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:56.759831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ff475-146e-4b38-9519-3080da6ca637 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:56.772867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 896/1763] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:56.787190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f297cc8-f46e-49d1-80e6-0a05ee048bf6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:56.789565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715fbac-3d7e-4091-81bd-05280d4757b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:56.794851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fbac-3d7e-4091-81bd-05280d4757b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:56.795184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fbac-3d7e-4091-81bd-05280d4757b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:56.795480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fbac-3d7e-4091-81bd-05280d4757b3 admin admin] PATH_INFO: `/v3/projects/07622697f04e4cebb8aa3a233c62d609/users/abb64038a84d4341bbabf552fe028b8b/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:56.840204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f297cc8-f46e-49d1-80e6-0a05ee048bf6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:56.853204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef397f7-2d66-4362-b991-35ecfe1b203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:56.854135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef397f7-2d66-4362-b991-35ecfe1b203d admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:56.854421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef397f7-2d66-4362-b991-35ecfe1b203d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:56.854700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef397f7-2d66-4362-b991-35ecfe1b203d admin admin] PATH_INFO: `/v3/projects/1c5d9fca2e5748c5b6b95a9203c9523b/users/faaf870aee544eddb9a079bceff6f9a2/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:56.891661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef397f7-2d66-4362-b991-35ecfe1b203d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5d9fca2e5748c5b6b95a9203c9523b, user_id=faaf870aee544eddb9a079bceff6f9a2, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:56.899119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715fbac-3d7e-4091-81bd-05280d4757b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07622697f04e4cebb8aa3a233c62d609, user_id=abb64038a84d4341bbabf552fe028b8b, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:56.901401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef397f7-2d66-4362-b991-35ecfe1b203d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:56.922478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715fbac-3d7e-4091-81bd-05280d4757b3 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:57.051158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 897/1764] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:56 2022] PUT /identity/v3/projects/1c5d9fca2e5748c5b6b95a9203c9523b/users/faaf870aee544eddb9a079bceff6f9a2/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:57.066178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a48ae9-cbaa-41c9-aa3b-b8531e9170bf None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.066484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a48ae9-cbaa-41c9-aa3b-b8531e9170bf None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.066769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a48ae9-cbaa-41c9-aa3b-b8531e9170bf None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.068179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 898/1765] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:34:57 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:34:57.070067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a48ae9-cbaa-41c9-aa3b-b8531e9170bf None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:57.073285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 868/1766] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:56 2022] PUT /identity/v3/projects/07622697f04e4cebb8aa3a233c62d609/users/abb64038a84d4341bbabf552fe028b8b/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:57.075098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715fbac-3d7e-4091-81bd-05280d4757b3 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:57.154437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715fbac-3d7e-4091-81bd-05280d4757b3 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:57.160279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc53ed-9c8d-43e7-8f2a-337af09f1ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:57.161316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc53ed-9c8d-43e7-8f2a-337af09f1ccc admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.173256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc53ed-9c8d-43e7-8f2a-337af09f1ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.174842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc53ed-9c8d-43e7-8f2a-337af09f1ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.178126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc53ed-9c8d-43e7-8f2a-337af09f1ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:57.179987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc53ed-9c8d-43e7-8f2a-337af09f1ccc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:57.221519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 869/1767] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:57.225893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9940-900d-45eb-abd6-bae831b41bda None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.226199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9940-900d-45eb-abd6-bae831b41bda None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.226491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9940-900d-45eb-abd6-bae831b41bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.275433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6abc9940-900d-45eb-abd6-bae831b41bda None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:57.275628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a48ae9-cbaa-41c9-aa3b-b8531e9170bf None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:57.277218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6abc9940-900d-45eb-abd6-bae831b41bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:57.291333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9bba8-5e92-4268-8544-70c765af2074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:57.292042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9bba8-5e92-4268-8544-70c765af2074 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.292335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9bba8-5e92-4268-8544-70c765af2074 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.292644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9bba8-5e92-4268-8544-70c765af2074 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.293397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9bba8-5e92-4268-8544-70c765af2074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:57.295010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9bba8-5e92-4268-8544-70c765af2074 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:57.323223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 899/1768] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:57.324213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9bba8-5e92-4268-8544-70c765af2074 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:57.351045 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9bba8-5e92-4268-8544-70c765af2074 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:57.359434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbfcd5-5953-4172-ad2e-bef39fc5aeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:57.364683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbfcd5-5953-4172-ad2e-bef39fc5aeac admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.364683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbfcd5-5953-4172-ad2e-bef39fc5aeac admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.364683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbfcd5-5953-4172-ad2e-bef39fc5aeac admin admin] PATH_INFO: `/v3/projects/07622697f04e4cebb8aa3a233c62d609/users/abb64038a84d4341bbabf552fe028b8b/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.371472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbfcd5-5953-4172-ad2e-bef39fc5aeac admin admin] RBAC: Authorizing `identity:create_grant(project_id=07622697f04e4cebb8aa3a233c62d609, user_id=abb64038a84d4341bbabf552fe028b8b, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:57.376416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbfcd5-5953-4172-ad2e-bef39fc5aeac admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:57.409548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 900/1769] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:57 2022] PUT /identity/v3/projects/07622697f04e4cebb8aa3a233c62d609/users/abb64038a84d4341bbabf552fe028b8b/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:57.413678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbfcd5-5953-4172-ad2e-bef39fc5aeac admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:57.431970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 870/1770] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:57.433684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5912f1-a0be-4b4e-9476-cc42b2ed4060 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.434058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5912f1-a0be-4b4e-9476-cc42b2ed4060 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.444894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5912f1-a0be-4b4e-9476-cc42b2ed4060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.542879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8a5912f1-a0be-4b4e-9476-cc42b2ed4060 None None] MFA Rules not processed for user `abb64038a84d4341bbabf552fe028b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:57.544134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8a5912f1-a0be-4b4e-9476-cc42b2ed4060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:57.665009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbfcd5-5953-4172-ad2e-bef39fc5aeac admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:57.676908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:57.678584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.678975 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.679339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] PATH_INFO: `/v3/projects/1c5d9fca2e5748c5b6b95a9203c9523b/users/faaf870aee544eddb9a079bceff6f9a2/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.682965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5d9fca2e5748c5b6b95a9203c9523b, user_id=faaf870aee544eddb9a079bceff6f9a2, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:57.686488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:57.713239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 901/1771] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:57 2022] PUT /identity/v3/projects/1c5d9fca2e5748c5b6b95a9203c9523b/users/faaf870aee544eddb9a079bceff6f9a2/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:57.715617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 871/1772] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:57.717871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:57.724843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:57.734658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332fbb-7606-40fb-9da4-2f220443d1e8 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.736073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332fbb-7606-40fb-9da4-2f220443d1e8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.737340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332fbb-7606-40fb-9da4-2f220443d1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.809250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a5332fbb-7606-40fb-9da4-2f220443d1e8 None None] MFA Rules not processed for user `faaf870aee544eddb9a079bceff6f9a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:57.810587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a5332fbb-7606-40fb-9da4-2f220443d1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:57.881930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484160dc-4ddc-4f7c-a97d-a80d0d2f4602 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:57.896251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:57.897617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:57.897789 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:57.898077 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:57.910157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:57.917422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:57.937081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 902/1773] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:57.938021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:57.940390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:57.999118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 872/1774] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:58.009904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5332fbb-7606-40fb-9da4-2f220443d1e8 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:58.117879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5332fbb-7606-40fb-9da4-2f220443d1e8 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:58.130742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102bddf-24b1-4bb3-8e28-40a781908d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:58.132847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102bddf-24b1-4bb3-8e28-40a781908d83 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:58.135137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102bddf-24b1-4bb3-8e28-40a781908d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:58.136754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102bddf-24b1-4bb3-8e28-40a781908d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:58.162388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102bddf-24b1-4bb3-8e28-40a781908d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:58.164366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102bddf-24b1-4bb3-8e28-40a781908d83 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:58.175694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58bc10-12b6-4a1d-b206-03baf19906cc service placement] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:58.185271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5ac2-f791-49f2-9a20-1f993f76178c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:58.186248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5ac2-f791-49f2-9a20-1f993f76178c service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:58.186802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5ac2-f791-49f2-9a20-1f993f76178c service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:58.187240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5ac2-f791-49f2-9a20-1f993f76178c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:58.204111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 873/1775] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:58.218502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a86c0-ef69-4024-a605-717b5c2e236a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:58.218829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a86c0-ef69-4024-a605-717b5c2e236a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:58.219116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a86c0-ef69-4024-a605-717b5c2e236a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:58.243350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-054a86c0-ef69-4024-a605-717b5c2e236a None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:58.244070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-054a86c0-ef69-4024-a605-717b5c2e236a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:58.270084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:58.328477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa5ac2-f791-49f2-9a20-1f993f76178c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:58.330681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa5ac2-f791-49f2-9a20-1f993f76178c service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:58.354337 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 903/1776] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:58.362854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 874/1777] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:34:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:58.572988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5ac2-f791-49f2-9a20-1f993f76178c service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:58.600589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5ac2-f791-49f2-9a20-1f993f76178c service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:58.611317 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe503446-d0ce-40f6-b823-118b5f2869a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:58.614961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe503446-d0ce-40f6-b823-118b5f2869a3 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:58.615269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe503446-d0ce-40f6-b823-118b5f2869a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:58.615557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe503446-d0ce-40f6-b823-118b5f2869a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:58.744606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe503446-d0ce-40f6-b823-118b5f2869a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:58.746617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe503446-d0ce-40f6-b823-118b5f2869a3 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:58.776870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 904/1778] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:58.879557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:58.966041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a86c0-ef69-4024-a605-717b5c2e236a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:58.967892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-054a86c0-ef69-4024-a605-717b5c2e236a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:58.975188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a038a5-bf02-4688-9626-bf02fa90d3f1 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:58.975669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a038a5-bf02-4688-9626-bf02fa90d3f1 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:58.975983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a038a5-bf02-4688-9626-bf02fa90d3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:59.014099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c8a038a5-bf02-4688-9626-bf02fa90d3f1 None None] MFA Rules not processed for user `c9dcfb252f0d44a280b3448094883e13`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:59.015360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c8a038a5-bf02-4688-9626-bf02fa90d3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:59.083141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a86c0-ef69-4024-a605-717b5c2e236a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:59.114933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:59.117755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:59.120121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:59.120767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:59.125765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 905/1779] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:34:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 00:34:59.127186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a1d54-d1fe-4e9d-8f1c-a491cfd2f354 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:59.135821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a1d54-d1fe-4e9d-8f1c-a491cfd2f354 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:59.135821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a1d54-d1fe-4e9d-8f1c-a491cfd2f354 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:59.135821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 906/1780] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:34:59.149118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1df4c9-c0a9-4ea7-8640-09e9e9775a28 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:59.149423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1df4c9-c0a9-4ea7-8640-09e9e9775a28 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:59.149709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1df4c9-c0a9-4ea7-8640-09e9e9775a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:59.169979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:59.176574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:59.197785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 875/1781] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:34:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:34:59.198976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:59.199131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-9c1df4c9-c0a9-4ea7-8640-09e9e9775a28 None None] MFA Rules not processed for user `acc54fd2a9dd44b6b658c44edf170e83`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:59.202781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:59.204936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9c1df4c9-c0a9-4ea7-8640-09e9e9775a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:59.224878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 907/1782] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:34:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:59.259630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1df4c9-c0a9-4ea7-8640-09e9e9775a28 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:59.287282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1df4c9-c0a9-4ea7-8640-09e9e9775a28 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:59.316861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad8685-88b4-4af1-bb07-11a328622e2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:59.317538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8685-88b4-4af1-bb07-11a328622e2d service placement] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:59.317824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8685-88b4-4af1-bb07-11a328622e2d service placement] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:59.318102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8685-88b4-4af1-bb07-11a328622e2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:59.343918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad8685-88b4-4af1-bb07-11a328622e2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:59.345580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad8685-88b4-4af1-bb07-11a328622e2d service placement] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:59.368475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 908/1783] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:59.394281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5b16e-b0d7-4158-b4f1-2c62b5239242 service swift] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:59.409441 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:59.413959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:59.413959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:59.413959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] PATH_INFO: `/v3/users/0c0133a208114aa0894aaa6b12ef07b6` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:59.414684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0133a208114aa0894aaa6b12ef07b6)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:59.417396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:59.739902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0133a208114aa0894aaa6b12ef07b6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.776385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0133a208114aa0894aaa6b12ef07b6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.777622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0133a208114aa0894aaa6b12ef07b6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.780309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0133a208114aa0894aaa6b12ef07b6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.861493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0133a208114aa0894aaa6b12ef07b6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.901562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 876/1784] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:59 2022] DELETE /identity/v3/users/0c0133a208114aa0894aaa6b12ef07b6 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:00.294180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:01.100830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad8685-88b4-4af1-bb07-11a328622e2d service placement] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.126024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad8685-88b4-4af1-bb07-11a328622e2d service placement] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.168987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.170457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:01.321178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea369-5389-47ce-97c5-94b3140fc422 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.322575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.322575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.322575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.322575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] PATH_INFO: `/v3/projects/db99dfcca0a54888a1298320b149323d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.322982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=db99dfcca0a54888a1298320b149323d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.322982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.328698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186174ed-a897-4f9c-aa5f-439def09f977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.329378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186174ed-a897-4f9c-aa5f-439def09f977 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.329709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186174ed-a897-4f9c-aa5f-439def09f977 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.329947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186174ed-a897-4f9c-aa5f-439def09f977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.339284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-186174ed-a897-4f9c-aa5f-439def09f977 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:01.439297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:01.625950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186174ed-a897-4f9c-aa5f-439def09f977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.636234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186174ed-a897-4f9c-aa5f-439def09f977 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.673266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db99dfcca0a54888a1298320b149323d', 'request_id': 'req-f87e2508-6a38-45e5-8f2a-b154c024ce7d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.708086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 877/1785] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:01.709732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afc37b-3429-459c-b478-f291e58b910a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.711246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afc37b-3429-459c-b478-f291e58b910a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.713258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afc37b-3429-459c-b478-f291e58b910a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.757650 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f2afc37b-3429-459c-b478-f291e58b910a None None] MFA Rules not processed for user `96df1ee2cbc14bcc8a8a250fd8f349dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:01.764017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f2afc37b-3429-459c-b478-f291e58b910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:01.841784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:01.842641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db99dfcca0a54888a1298320b149323d', 'request_id': 'req-f87e2508-6a38-45e5-8f2a-b154c024ce7d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.858994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db99dfcca0a54888a1298320b149323d', 'request_id': 'req-f87e2508-6a38-45e5-8f2a-b154c024ce7d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.906434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] The token cache is being invalidate because project db99dfcca0a54888a1298320b149323d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:01.915635 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.918894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f87e2508-6a38-45e5-8f2a-b154c024ce7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.929008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 909/1786] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:01 2022] DELETE /identity/v3/projects/db99dfcca0a54888a1298320b149323d => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:01.938640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d265-d996-40c6-b8dc-3adcdbd62df4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.941378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d265-d996-40c6-b8dc-3adcdbd62df4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.946059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d265-d996-40c6-b8dc-3adcdbd62df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.996103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 878/1787] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jun 16 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:01.998480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afc37b-3429-459c-b478-f291e58b910a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.999825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6964d265-d996-40c6-b8dc-3adcdbd62df4 None None] MFA Rules not processed for user `31973e5a04c14cfc90ab377eac5f4194`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:02.001159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f2afc37b-3429-459c-b478-f291e58b910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:02.009657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6964d265-d996-40c6-b8dc-3adcdbd62df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:02.163999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 910/1788] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:02.165595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964d265-d996-40c6-b8dc-3adcdbd62df4 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.170473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6964d265-d996-40c6-b8dc-3adcdbd62df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:02.197009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964d265-d996-40c6-b8dc-3adcdbd62df4 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.206980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afc37b-3429-459c-b478-f291e58b910a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.209693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021867-5a8d-49cb-bf6a-348156b01cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.210434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021867-5a8d-49cb-bf6a-348156b01cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.210956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021867-5a8d-49cb-bf6a-348156b01cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.211445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021867-5a8d-49cb-bf6a-348156b01cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.217585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.218732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.219023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.219302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] PATH_INFO: `/v3/users/6744bbcd367646cfbe78ca3bc4b30ae2` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.220636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6744bbcd367646cfbe78ca3bc4b30ae2)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.225319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.236034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021867-5a8d-49cb-bf6a-348156b01cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.238493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021867-5a8d-49cb-bf6a-348156b01cb8 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.256563 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 911/1789] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:02.638400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6744bbcd367646cfbe78ca3bc4b30ae2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.690448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6744bbcd367646cfbe78ca3bc4b30ae2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.696488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6744bbcd367646cfbe78ca3bc4b30ae2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.699689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6744bbcd367646cfbe78ca3bc4b30ae2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.849294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-96955e43-2b32-4597-ac42-a2784381ea87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6744bbcd367646cfbe78ca3bc4b30ae2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.888913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 879/1790] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:01 2022] DELETE /identity/v3/users/6744bbcd367646cfbe78ca3bc4b30ae2 => generated 0 bytes in 892 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:02.932420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:03.377957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:04.649600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:04.691237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021867-5a8d-49cb-bf6a-348156b01cb8 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:04.733039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021867-5a8d-49cb-bf6a-348156b01cb8 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:04.974357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:04.975286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:04.975559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:04.975847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] PATH_INFO: `/v3/projects/1c44371d0bec44599ffd0d7d54b504ad` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:04.984291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c44371d0bec44599ffd0d7d54b504ad)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:04.989115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:05.022187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9efb3-0c67-45dd-ba6d-b6e98e58db1c None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.022474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9efb3-0c67-45dd-ba6d-b6e98e58db1c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.027007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9efb3-0c67-45dd-ba6d-b6e98e58db1c None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.027007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 880/1791] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:35:05.090231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94e156-2d97-4580-90b8-99dc4cc2a21a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.090754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94e156-2d97-4580-90b8-99dc4cc2a21a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.091153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94e156-2d97-4580-90b8-99dc4cc2a21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.101124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:05.159460 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ff94e156-2d97-4580-90b8-99dc4cc2a21a None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:05.164412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ff94e156-2d97-4580-90b8-99dc4cc2a21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:05.221374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c44371d0bec44599ffd0d7d54b504ad', 'request_id': 'req-5062457b-2799-488f-8ea2-b32c41bd93dd'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.316775 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c44371d0bec44599ffd0d7d54b504ad', 'request_id': 'req-5062457b-2799-488f-8ea2-b32c41bd93dd'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.318376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:05.326025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c44371d0bec44599ffd0d7d54b504ad', 'request_id': 'req-5062457b-2799-488f-8ea2-b32c41bd93dd'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.356265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] The token cache is being invalidate because project 1c44371d0bec44599ffd0d7d54b504ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:05.357541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.360963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5062457b-2799-488f-8ea2-b32c41bd93dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.365098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 912/1792] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:04 2022] DELETE /identity/v3/projects/1c44371d0bec44599ffd0d7d54b504ad => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:05.373251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df158cc-6461-43f3-aafd-ca6b8a485965 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.373855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df158cc-6461-43f3-aafd-ca6b8a485965 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.375285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df158cc-6461-43f3-aafd-ca6b8a485965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.431732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4df158cc-6461-43f3-aafd-ca6b8a485965 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:05.438007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4df158cc-6461-43f3-aafd-ca6b8a485965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:05.571155 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 881/1793] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:35:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:05.622830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 913/1794] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:05.633387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7b432-577f-4188-b718-c1115b5ead1f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.633923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7b432-577f-4188-b718-c1115b5ead1f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.634403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7b432-577f-4188-b718-c1115b5ead1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.687297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9bf7b432-577f-4188-b718-c1115b5ead1f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:05.689976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9bf7b432-577f-4188-b718-c1115b5ead1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:05.711493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 882/1795] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:05.715138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df158cc-6461-43f3-aafd-ca6b8a485965 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:05.749042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df158cc-6461-43f3-aafd-ca6b8a485965 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:05.764554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:05.765534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.766143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.766674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.768351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:05.769034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:05.771396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:05.795784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7b432-577f-4188-b718-c1115b5ead1f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:05.802748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 914/1796] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:05.807003 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7b432-577f-4188-b718-c1115b5ead1f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:05.813714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8faaa4-fa61-4e59-8ca0-d6d781be7d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:05.814973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8faaa4-fa61-4e59-8ca0-d6d781be7d8d admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.815420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8faaa4-fa61-4e59-8ca0-d6d781be7d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.815856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8faaa4-fa61-4e59-8ca0-d6d781be7d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.821374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8faaa4-fa61-4e59-8ca0-d6d781be7d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:05.821374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8faaa4-fa61-4e59-8ca0-d6d781be7d8d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:05.985821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:05.997112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 883/1797] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:05 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:06.003787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936a9ee-17fc-4b0d-be5a-9fa05abebe24 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.141438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:06.159421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a3e4e-d6a7-4c3a-9b20-f7afe5a23eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.159421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3e4e-d6a7-4c3a-9b20-f7afe5a23eeb admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.159421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3e4e-d6a7-4c3a-9b20-f7afe5a23eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.159421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3e4e-d6a7-4c3a-9b20-f7afe5a23eeb admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.159840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a3e4e-d6a7-4c3a-9b20-f7afe5a23eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.159840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a3e4e-d6a7-4c3a-9b20-f7afe5a23eeb admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.273795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 915/1798] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:05 2022] POST /identity/v3/users => generated 287 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:06.277916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8faaa4-fa61-4e59-8ca0-d6d781be7d8d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.291637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8faaa4-fa61-4e59-8ca0-d6d781be7d8d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.303002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.309752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.309752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.309752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.309752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.309752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.336766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 884/1799] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:06.337271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a3e4e-d6a7-4c3a-9b20-f7afe5a23eeb admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.351160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a3e4e-d6a7-4c3a-9b20-f7afe5a23eeb admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.382466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dd124-2e4c-4bc2-9c59-e2289477bc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.382466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd124-2e4c-4bc2-9c59-e2289477bc9b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.382466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd124-2e4c-4bc2-9c59-e2289477bc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.382466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd124-2e4c-4bc2-9c59-e2289477bc9b admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.383109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dd124-2e4c-4bc2-9c59-e2289477bc9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.383109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dd124-2e4c-4bc2-9c59-e2289477bc9b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.392397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.397235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.403153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 916/1800] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:06.411462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dd124-2e4c-4bc2-9c59-e2289477bc9b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.428850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dd124-2e4c-4bc2-9c59-e2289477bc9b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.439271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd8f2-456e-4e64-a064-342aff17724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.441010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd8f2-456e-4e64-a064-342aff17724d admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.441450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd8f2-456e-4e64-a064-342aff17724d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.441914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd8f2-456e-4e64-a064-342aff17724d admin admin] PATH_INFO: `/v3/projects/79149fa8c19a432c97901f360b18a8c0/users/20aff2ad1d2449f998d0f92145d6da0f/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.468215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfd8f2-456e-4e64-a064-342aff17724d admin admin] RBAC: Authorizing `identity:create_grant(project_id=79149fa8c19a432c97901f360b18a8c0, user_id=20aff2ad1d2449f998d0f92145d6da0f, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.471684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfd8f2-456e-4e64-a064-342aff17724d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.520805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1505-cb93-4ebd-9b61-8c45227f7e5c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.530403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c0651-e957-461c-bb11-ab674f3908a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.531183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c0651-e957-461c-bb11-ab674f3908a2 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.531541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c0651-e957-461c-bb11-ab674f3908a2 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.531940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c0651-e957-461c-bb11-ab674f3908a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.542795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c0651-e957-461c-bb11-ab674f3908a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.553156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c0651-e957-461c-bb11-ab674f3908a2 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.556712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 885/1801] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:06.608882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 917/1802] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:06 2022] PUT /identity/v3/projects/79149fa8c19a432c97901f360b18a8c0/users/20aff2ad1d2449f998d0f92145d6da0f/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:06.619079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22fcf4-770f-4c2a-b4a7-2e6707688d2e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.619403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22fcf4-770f-4c2a-b4a7-2e6707688d2e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.619657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22fcf4-770f-4c2a-b4a7-2e6707688d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.671938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9e22fcf4-770f-4c2a-b4a7-2e6707688d2e None None] MFA Rules not processed for user `20aff2ad1d2449f998d0f92145d6da0f`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:06.683450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9e22fcf4-770f-4c2a-b4a7-2e6707688d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.771508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 886/1803] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:06.775756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd8f2-456e-4e64-a064-342aff17724d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.801911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:06.862350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd8f2-456e-4e64-a064-342aff17724d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.869565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dd0ff-1d01-4a27-9757-481194048745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.870515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd0ff-1d01-4a27-9757-481194048745 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.871227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd0ff-1d01-4a27-9757-481194048745 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.871801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd0ff-1d01-4a27-9757-481194048745 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.874199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dd0ff-1d01-4a27-9757-481194048745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.878232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dd0ff-1d01-4a27-9757-481194048745 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.014519 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 918/1804] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:06 2022] POST /identity/v3/projects => generated 371 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:07.022699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22fcf4-770f-4c2a-b4a7-2e6707688d2e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.044628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22fcf4-770f-4c2a-b4a7-2e6707688d2e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.195956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55def4-b0b7-4791-96ba-dcd05b6835f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.197183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55def4-b0b7-4791-96ba-dcd05b6835f6 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.197658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55def4-b0b7-4791-96ba-dcd05b6835f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.198159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55def4-b0b7-4791-96ba-dcd05b6835f6 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.199189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55def4-b0b7-4791-96ba-dcd05b6835f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.204969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55def4-b0b7-4791-96ba-dcd05b6835f6 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.318032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 887/1805] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:07 2022] POST /identity/v3/users => generated 292 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:07.322139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dd0ff-1d01-4a27-9757-481194048745 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.338989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dd0ff-1d01-4a27-9757-481194048745 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.348128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e0f18-1380-47d4-8b24-69847d667626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.348874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e0f18-1380-47d4-8b24-69847d667626 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.349220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e0f18-1380-47d4-8b24-69847d667626 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.349500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e0f18-1380-47d4-8b24-69847d667626 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.351252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e0f18-1380-47d4-8b24-69847d667626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.353153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e0f18-1380-47d4-8b24-69847d667626 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.366449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 919/1806] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:07.370772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55def4-b0b7-4791-96ba-dcd05b6835f6 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.383494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55def4-b0b7-4791-96ba-dcd05b6835f6 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.389242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6628e4-df18-446a-a209-e30838465701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.390509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6628e4-df18-446a-a209-e30838465701 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.390799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6628e4-df18-446a-a209-e30838465701 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.391080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6628e4-df18-446a-a209-e30838465701 admin admin] PATH_INFO: `/v3/projects/cead018c947641e4b83680e5609c78f0/users/75c538ee91874e21ab986ffc744080e7/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.409352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ddf53-d68f-4b84-901e-faadeb4d0f29 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.409352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ddf53-d68f-4b84-901e-faadeb4d0f29 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.409352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ddf53-d68f-4b84-901e-faadeb4d0f29 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.410231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 920/1807] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:35:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:35:07.414101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6628e4-df18-446a-a209-e30838465701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cead018c947641e4b83680e5609c78f0, user_id=75c538ee91874e21ab986ffc744080e7, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.418410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6628e4-df18-446a-a209-e30838465701 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.422913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104f25-1da6-4931-a875-10d5a1673288 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.423205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104f25-1da6-4931-a875-10d5a1673288 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.423485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104f25-1da6-4931-a875-10d5a1673288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.465770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a9104f25-1da6-4931-a875-10d5a1673288 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:07.467544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a9104f25-1da6-4931-a875-10d5a1673288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:07.594496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 888/1808] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:07 2022] PUT /identity/v3/projects/cead018c947641e4b83680e5609c78f0/users/75c538ee91874e21ab986ffc744080e7/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:07.595564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6628e4-df18-446a-a209-e30838465701 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.653668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 921/1809] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:35:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:07.665240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9104f25-1da6-4931-a875-10d5a1673288 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.669459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a9104f25-1da6-4931-a875-10d5a1673288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:07.690713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9104f25-1da6-4931-a875-10d5a1673288 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.707661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6628e4-df18-446a-a209-e30838465701 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.713343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750241a1-5385-4268-a554-3545b34c70df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.714035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750241a1-5385-4268-a554-3545b34c70df admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.716422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750241a1-5385-4268-a554-3545b34c70df admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.716422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750241a1-5385-4268-a554-3545b34c70df admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.716422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750241a1-5385-4268-a554-3545b34c70df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.717203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750241a1-5385-4268-a554-3545b34c70df admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.746411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 889/1810] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:07.747710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750241a1-5385-4268-a554-3545b34c70df admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.759123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750241a1-5385-4268-a554-3545b34c70df admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.764968 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f47f3-f440-4c23-b333-942c36bde285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.766047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f47f3-f440-4c23-b333-942c36bde285 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.766474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f47f3-f440-4c23-b333-942c36bde285 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.766895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f47f3-f440-4c23-b333-942c36bde285 admin admin] PATH_INFO: `/v3/projects/cead018c947641e4b83680e5609c78f0/users/75c538ee91874e21ab986ffc744080e7/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.770260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f47f3-f440-4c23-b333-942c36bde285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cead018c947641e4b83680e5609c78f0, user_id=75c538ee91874e21ab986ffc744080e7, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.773763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f47f3-f440-4c23-b333-942c36bde285 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.775665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.776343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.776965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.776965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.801021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.801021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.801426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 890/1811] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:07 2022] PUT /identity/v3/projects/cead018c947641e4b83680e5609c78f0/users/75c538ee91874e21ab986ffc744080e7/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:07.803542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0115e-19aa-4b69-a639-ab52394a591f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.803834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0115e-19aa-4b69-a639-ab52394a591f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.804130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0115e-19aa-4b69-a639-ab52394a591f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.816624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 922/1812] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:07.835716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6df0115e-19aa-4b69-a639-ab52394a591f None None] MFA Rules not processed for user `75c538ee91874e21ab986ffc744080e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:07.837185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6df0115e-19aa-4b69-a639-ab52394a591f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:07.919482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 891/1813] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:07.930817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.932221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:08.003950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f60065-130e-43f7-9380-18fd5c94e23d service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:08.042241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:08.042943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:08.043239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:08.043520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:08.056332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:08.056332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:08.065867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 923/1814] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:08.097804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:08.334784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:09.213735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0115e-19aa-4b69-a639-ab52394a591f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:09.270187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:09.286385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0115e-19aa-4b69-a639-ab52394a591f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:09.295325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:09.296618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:09.297125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:09.297981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] PATH_INFO: `/v3/users/20aff2ad1d2449f998d0f92145d6da0f` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:09.309784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=20aff2ad1d2449f998d0f92145d6da0f)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:09.309784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:09.572447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20aff2ad1d2449f998d0f92145d6da0f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:09.584217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20aff2ad1d2449f998d0f92145d6da0f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:09.585013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20aff2ad1d2449f998d0f92145d6da0f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:09.585928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20aff2ad1d2449f998d0f92145d6da0f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:09.660374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:09.662015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:09.674506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e5554ed2-3ba8-4ba9-9d49-7084a2f703ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20aff2ad1d2449f998d0f92145d6da0f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:09.680486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fd4a7-b11e-48b4-932b-14d630c132d8 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:09.687869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:09.688835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:09.689292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:09.689625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:09.692248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:09.697763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 892/1815] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:09 2022] DELETE /identity/v3/users/20aff2ad1d2449f998d0f92145d6da0f => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:09.702335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea534-c3e7-48d2-81c9-78fc5d65425d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:09.702678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea534-c3e7-48d2-81c9-78fc5d65425d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:09.703163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea534-c3e7-48d2-81c9-78fc5d65425d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:09.747022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ef2ea534-c3e7-48d2-81c9-78fc5d65425d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:09.748635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ef2ea534-c3e7-48d2-81c9-78fc5d65425d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:09.798910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:09.800421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:09.844312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 924/1816] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:09.846406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:09.867408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 893/1817] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:09.877535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:09.879815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:09.964160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d991576-f2fb-44ff-b460-78abf229a6b4 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:10.026967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435524f-0296-4f6c-9c79-3af2a502d826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:10.034711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435524f-0296-4f6c-9c79-3af2a502d826 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:10.034711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435524f-0296-4f6c-9c79-3af2a502d826 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:10.034711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435524f-0296-4f6c-9c79-3af2a502d826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:10.042604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435524f-0296-4f6c-9c79-3af2a502d826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:10.044297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435524f-0296-4f6c-9c79-3af2a502d826 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:10.056717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 925/1818] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:10.172566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:11.809139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea534-c3e7-48d2-81c9-78fc5d65425d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:11.810576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ef2ea534-c3e7-48d2-81c9-78fc5d65425d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:11.835252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea534-c3e7-48d2-81c9-78fc5d65425d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:11.850548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:11.851301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:11.851584 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:11.851858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] PATH_INFO: `/v3/projects/79149fa8c19a432c97901f360b18a8c0` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:11.856279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79149fa8c19a432c97901f360b18a8c0)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:11.858309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:12.001601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:12.041300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79149fa8c19a432c97901f360b18a8c0', 'request_id': 'req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.124320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79149fa8c19a432c97901f360b18a8c0', 'request_id': 'req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.140861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79149fa8c19a432c97901f360b18a8c0', 'request_id': 'req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.175380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] The token cache is being invalidate because project 79149fa8c19a432c97901f360b18a8c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:12.176135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.176996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.178799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 894/1819] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:11 2022] DELETE /identity/v3/projects/79149fa8c19a432c97901f360b18a8c0 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:12.183285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435524f-0296-4f6c-9c79-3af2a502d826 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:12.229126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435524f-0296-4f6c-9c79-3af2a502d826 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:12.279339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:12.431181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:12.432170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:12.432623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:12.432962 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] PATH_INFO: `/v3/users/75c538ee91874e21ab986ffc744080e7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:12.443188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c538ee91874e21ab986ffc744080e7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:12.446508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:12.709876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c538ee91874e21ab986ffc744080e7'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.745880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c538ee91874e21ab986ffc744080e7'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.758277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c538ee91874e21ab986ffc744080e7'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.759376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c538ee91874e21ab986ffc744080e7'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.760835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-26860b3b-a01e-45d7-9abd-ff33e0efae63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c538ee91874e21ab986ffc744080e7'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.888304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 926/1820] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:12 2022] DELETE /identity/v3/users/75c538ee91874e21ab986ffc744080e7 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:13.599264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:14.627814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:14.629680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:14.656087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f6bb7-d1c0-4b6d-bd15-68e76b9178d3 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:14.813006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:14.815097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:14.815656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:14.816416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] PATH_INFO: `/v3/projects/cead018c947641e4b83680e5609c78f0` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:14.818687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cead018c947641e4b83680e5609c78f0)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:14.822331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:14.920052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:15.342150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cead018c947641e4b83680e5609c78f0', 'request_id': 'req-35423cf3-c394-4538-a31d-a16c55ee3624'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:15.434599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cead018c947641e4b83680e5609c78f0', 'request_id': 'req-35423cf3-c394-4538-a31d-a16c55ee3624'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:15.445570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cead018c947641e4b83680e5609c78f0', 'request_id': 'req-35423cf3-c394-4538-a31d-a16c55ee3624'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:15.460480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] The token cache is being invalidate because project cead018c947641e4b83680e5609c78f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:15.460845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:15.461987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:15.469175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b488a-a452-4d7f-899b-2d73cfc93c93 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:15.473121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b488a-a452-4d7f-899b-2d73cfc93c93 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:15.473121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b488a-a452-4d7f-899b-2d73cfc93c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:15.473301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 895/1821] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:14 2022] DELETE /identity/v3/projects/cead018c947641e4b83680e5609c78f0 => generated 0 bytes in 837 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:15.495143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-546b488a-a452-4d7f-899b-2d73cfc93c93 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:15.496759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-546b488a-a452-4d7f-899b-2d73cfc93c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:15.574656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:15.576124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:15.615851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 927/1822] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:15.617259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b819d7-2888-4669-b872-d0767182ad3f None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:15.617807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b819d7-2888-4669-b872-d0767182ad3f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:15.618299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b819d7-2888-4669-b872-d0767182ad3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:15.644367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-93b819d7-2888-4669-b872-d0767182ad3f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:15.645976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-93b819d7-2888-4669-b872-d0767182ad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:15.654876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 928/1823] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:15.659998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b819d7-2888-4669-b872-d0767182ad3f None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:15.665570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35423cf3-c394-4538-a31d-a16c55ee3624 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:15.672171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b819d7-2888-4669-b872-d0767182ad3f None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:15.673841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:15.674382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:15.675009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:15.675588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] PATH_INFO: `/v3/users/dceab1b1494b43d0a936e9d0237e9da4` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:15.677152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dceab1b1494b43d0a936e9d0237e9da4)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:15.677668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:15.678436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:15.678726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:15.679440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:15.679555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:15.679885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:15.680203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:15.682496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:15.704723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 929/1824] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:35:15.707286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:15.717757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d9e7a-a43e-415b-97b7-97575db8b6ea admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:15.723463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfc0d3-9ddb-4b57-9ecd-39b744667907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:15.725387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc0d3-9ddb-4b57-9ecd-39b744667907 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:15.725387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc0d3-9ddb-4b57-9ecd-39b744667907 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:15.725387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc0d3-9ddb-4b57-9ecd-39b744667907 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:15.725387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfc0d3-9ddb-4b57-9ecd-39b744667907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:15.727307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfc0d3-9ddb-4b57-9ecd-39b744667907 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:15.840319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 930/1825] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:15 2022] POST /identity/v3/projects => generated 393 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:15.844268 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfc0d3-9ddb-4b57-9ecd-39b744667907 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:15.942673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dceab1b1494b43d0a936e9d0237e9da4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:15.976033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dceab1b1494b43d0a936e9d0237e9da4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:15.976033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dceab1b1494b43d0a936e9d0237e9da4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:15.976033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dceab1b1494b43d0a936e9d0237e9da4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:16.019203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfc0d3-9ddb-4b57-9ecd-39b744667907 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:16.024995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:16.026190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:16.026587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:16.027299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:16.028253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:16.031164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:16.063209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-a021365e-6912-4a94-8b73-4d7cd8c53402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dceab1b1494b43d0a936e9d0237e9da4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:16.085121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 896/1826] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:15 2022] DELETE /identity/v3/users/dceab1b1494b43d0a936e9d0237e9da4 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:16.089555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6cc7f-e4fd-461d-a886-06e84b48eb5c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:16.090024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6cc7f-e4fd-461d-a886-06e84b48eb5c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:16.091004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6cc7f-e4fd-461d-a886-06e84b48eb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:16.119302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-09c6cc7f-e4fd-461d-a886-06e84b48eb5c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:16.120698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-09c6cc7f-e4fd-461d-a886-06e84b48eb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:16.139322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 931/1827] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:15 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:16.143751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:16.149124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:16.165850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb53aa8-7cbc-46b5-bf06-5be9dfa29bd0 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:16.203617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 897/1828] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:16.221949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6cc7f-e4fd-461d-a886-06e84b48eb5c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:16.224033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-09c6cc7f-e4fd-461d-a886-06e84b48eb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:16.252727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491868b8-0be2-40f9-95ce-e8934379d278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:16.252727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491868b8-0be2-40f9-95ce-e8934379d278 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:16.252727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491868b8-0be2-40f9-95ce-e8934379d278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:16.252727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491868b8-0be2-40f9-95ce-e8934379d278 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:16.253754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491868b8-0be2-40f9-95ce-e8934379d278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:16.255228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491868b8-0be2-40f9-95ce-e8934379d278 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:16.271892 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 932/1829] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:16.274741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491868b8-0be2-40f9-95ce-e8934379d278 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:16.286872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491868b8-0be2-40f9-95ce-e8934379d278 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:16.291777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c5c3-7683-493d-a053-5a4777f33bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:16.292536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c5c3-7683-493d-a053-5a4777f33bed admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:16.292845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c5c3-7683-493d-a053-5a4777f33bed admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:16.293130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c5c3-7683-493d-a053-5a4777f33bed admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:16.293819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6cc7f-e4fd-461d-a886-06e84b48eb5c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:16.293966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333c5c3-7683-493d-a053-5a4777f33bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:16.296113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333c5c3-7683-493d-a053-5a4777f33bed admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:16.298196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ab2f-fe61-4ced-b6a9-fdadb0cfadc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:16.300447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ab2f-fe61-4ced-b6a9-fdadb0cfadc3 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:16.300447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ab2f-fe61-4ced-b6a9-fdadb0cfadc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:16.300447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ab2f-fe61-4ced-b6a9-fdadb0cfadc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:16.313488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 933/1830] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:16.314425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c5c3-7683-493d-a053-5a4777f33bed admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:16.316840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ab2f-fe61-4ced-b6a9-fdadb0cfadc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:16.318389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ab2f-fe61-4ced-b6a9-fdadb0cfadc3 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:16.326099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c5c3-7683-493d-a053-5a4777f33bed admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:16.332133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa3fc3-9c4e-4b6f-a757-8e8347b0cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:16.333850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa3fc3-9c4e-4b6f-a757-8e8347b0cc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:16.333850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa3fc3-9c4e-4b6f-a757-8e8347b0cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:16.333850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa3fc3-9c4e-4b6f-a757-8e8347b0cc03 admin admin] PATH_INFO: `/v3/projects/577c1a867889439782a4154e52064520/users/8be5459f856044ce9d40fb75aa9dc8af/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:16.357037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 898/1831] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:16.388061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa3fc3-9c4e-4b6f-a757-8e8347b0cc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=577c1a867889439782a4154e52064520, user_id=8be5459f856044ce9d40fb75aa9dc8af, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:16.390998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa3fc3-9c4e-4b6f-a757-8e8347b0cc03 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:16.432210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:16.503501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 934/1832] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:16 2022] PUT /identity/v3/projects/577c1a867889439782a4154e52064520/users/8be5459f856044ce9d40fb75aa9dc8af/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:16.513759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316f4e2-955a-4a7f-a438-a0ff8ba9626d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:16.514324 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316f4e2-955a-4a7f-a438-a0ff8ba9626d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:16.514449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316f4e2-955a-4a7f-a438-a0ff8ba9626d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:16.549537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e316f4e2-955a-4a7f-a438-a0ff8ba9626d None None] MFA Rules not processed for user `8be5459f856044ce9d40fb75aa9dc8af`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:16.550798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e316f4e2-955a-4a7f-a438-a0ff8ba9626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:16.637844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 899/1833] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:16.643398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa3fc3-9c4e-4b6f-a757-8e8347b0cc03 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:16.762334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa3fc3-9c4e-4b6f-a757-8e8347b0cc03 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:16.769971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:16.770768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:16.771058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:16.771348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:16.774566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:16.780190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:16.843641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:16.888009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 935/1834] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:16 2022] POST /identity/v3/projects => generated 393 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:16.892183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316f4e2-955a-4a7f-a438-a0ff8ba9626d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:16.905480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316f4e2-955a-4a7f-a438-a0ff8ba9626d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.050646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:17.052420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:17.064516 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e253e-d044-4d6b-ab95-0166844fa243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.066904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e253e-d044-4d6b-ab95-0166844fa243 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.068148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e253e-d044-4d6b-ab95-0166844fa243 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.068553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e253e-d044-4d6b-ab95-0166844fa243 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.069714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e253e-d044-4d6b-ab95-0166844fa243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:17.078239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e253e-d044-4d6b-ab95-0166844fa243 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:17.079229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69671c55-83c4-4fe1-8ba2-9d09016d8674 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.097285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.098376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.099692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.100186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] PATH_INFO: `/v3/projects/513f80928bb34def978c26254b5152f7` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.103503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=513f80928bb34def978c26254b5152f7)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:17.107685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:17.233281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 900/1835] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:16 2022] POST /identity/v3/users => generated 303 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:17.237916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e253e-d044-4d6b-ab95-0166844fa243 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:17.240739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513f80928bb34def978c26254b5152f7', 'request_id': 'req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.254462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e253e-d044-4d6b-ab95-0166844fa243 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.305788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5077cc1-1679-4254-a9a5-1ee8089bcb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.306470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5077cc1-1679-4254-a9a5-1ee8089bcb65 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.306760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5077cc1-1679-4254-a9a5-1ee8089bcb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.307046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5077cc1-1679-4254-a9a5-1ee8089bcb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.308024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5077cc1-1679-4254-a9a5-1ee8089bcb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:17.309745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5077cc1-1679-4254-a9a5-1ee8089bcb65 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:17.320433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 901/1836] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:17.324456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5077cc1-1679-4254-a9a5-1ee8089bcb65 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:17.334496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5077cc1-1679-4254-a9a5-1ee8089bcb65 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.337279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513f80928bb34def978c26254b5152f7', 'request_id': 'req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.339282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.340190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.340465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.346456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] PATH_INFO: `/v3/projects/8101fcbce6304257bbb63661d2b6a1e4/users/8efcb2a11c784cdfa3957703eeb893bb/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.346584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513f80928bb34def978c26254b5152f7', 'request_id': 'req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.356655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8101fcbce6304257bbb63661d2b6a1e4, user_id=8efcb2a11c784cdfa3957703eeb893bb, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:17.356800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] The token cache is being invalidate because project 513f80928bb34def978c26254b5152f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:17.357182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.357860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.359299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:17.359415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 936/1837] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:17 2022] DELETE /identity/v3/projects/513f80928bb34def978c26254b5152f7 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:17.363431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:17.364827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:17.377731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f1aa6-5730-4bd0-8991-c8ef9e4348a3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.382573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.383472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.383763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.384046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] PATH_INFO: `/v3/users/2e5a72e163bd4a818ad0502e7cdc8905` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.385261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5a72e163bd4a818ad0502e7cdc8905)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:17.387172 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:17.441799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 902/1838] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:17 2022] PUT /identity/v3/projects/8101fcbce6304257bbb63661d2b6a1e4/users/8efcb2a11c784cdfa3957703eeb893bb/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:17.445561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:17.449740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:17.514980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e887089-3d7c-4ec4-9dca-3e7cec1a1769 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.520422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2543c-8446-4af7-8389-368869dce8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.521124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2543c-8446-4af7-8389-368869dce8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.521412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2543c-8446-4af7-8389-368869dce8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.521692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2543c-8446-4af7-8389-368869dce8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.522425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2543c-8446-4af7-8389-368869dce8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:17.524274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2543c-8446-4af7-8389-368869dce8a1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:17.537744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 903/1839] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:17.538747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2543c-8446-4af7-8389-368869dce8a1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:17.548327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2543c-8446-4af7-8389-368869dce8a1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.552963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb7ce8-9892-4b40-91f5-783e50f401d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.553832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7ce8-9892-4b40-91f5-783e50f401d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.554116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7ce8-9892-4b40-91f5-783e50f401d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.554392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7ce8-9892-4b40-91f5-783e50f401d5 admin admin] PATH_INFO: `/v3/projects/8101fcbce6304257bbb63661d2b6a1e4/users/8efcb2a11c784cdfa3957703eeb893bb/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.556343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb7ce8-9892-4b40-91f5-783e50f401d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8101fcbce6304257bbb63661d2b6a1e4, user_id=8efcb2a11c784cdfa3957703eeb893bb, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:17.559114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb7ce8-9892-4b40-91f5-783e50f401d5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:17.570558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 904/1840] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:17 2022] PUT /identity/v3/projects/8101fcbce6304257bbb63661d2b6a1e4/users/8efcb2a11c784cdfa3957703eeb893bb/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:17.583380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6cd1-b524-455e-b68a-e6ceb7b6e7ea None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.583681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6cd1-b524-455e-b68a-e6ceb7b6e7ea None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.583977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6cd1-b524-455e-b68a-e6ceb7b6e7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.612433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d56e6cd1-b524-455e-b68a-e6ceb7b6e7ea None None] MFA Rules not processed for user `8efcb2a11c784cdfa3957703eeb893bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:17.613694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d56e6cd1-b524-455e-b68a-e6ceb7b6e7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:17.633513 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5a72e163bd4a818ad0502e7cdc8905'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.646771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5a72e163bd4a818ad0502e7cdc8905'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.653985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5a72e163bd4a818ad0502e7cdc8905'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.655076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5a72e163bd4a818ad0502e7cdc8905'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.655702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5a72e163bd4a818ad0502e7cdc8905'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:17.691018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 905/1841] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:17.701907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6cd1-b524-455e-b68a-e6ceb7b6e7ea None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:17.703308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d56e6cd1-b524-455e-b68a-e6ceb7b6e7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:17.758274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 937/1842] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:17 2022] DELETE /identity/v3/users/2e5a72e163bd4a818ad0502e7cdc8905 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:17.761705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6cd1-b524-455e-b68a-e6ceb7b6e7ea None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.825243 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.825914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.826485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.827377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.898744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:17.902722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:17.904190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:17.915125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 906/1843] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:17.973462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:17.976291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:17.990700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b141ac-a7f1-4c4d-85be-e7a317112f9a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:17.995435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:17.996234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:17.996765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:17.997121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:17.999282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:18.068448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:18.070006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:18.079362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 938/1844] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:18.457446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:18.459461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:18.541861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9353719-9dbb-4bcd-936d-437e81a1e39c service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:18.548810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:18.562812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:18.563281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:18.565018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] PATH_INFO: `/v3/projects/5633dda8fca54f9b84488004f2d2b050` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:18.569868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5633dda8fca54f9b84488004f2d2b050)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:18.573253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:18.786389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5633dda8fca54f9b84488004f2d2b050', 'request_id': 'req-9b8bcae8-a0be-420c-b331-89153119d783'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:18.926997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5633dda8fca54f9b84488004f2d2b050', 'request_id': 'req-9b8bcae8-a0be-420c-b331-89153119d783'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:18.935954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5633dda8fca54f9b84488004f2d2b050', 'request_id': 'req-9b8bcae8-a0be-420c-b331-89153119d783'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:18.949833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] The token cache is being invalidate because project 5633dda8fca54f9b84488004f2d2b050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:18.950433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:18.951242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:18.957869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 907/1845] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:18 2022] DELETE /identity/v3/projects/5633dda8fca54f9b84488004f2d2b050 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:18.960354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:18.978810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199cdd-332a-47c8-9f22-de256a11d8a3 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:19.037953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:19.072006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:19.073611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:19.074135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:19.074595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] PATH_INFO: `/v3/users/abb64038a84d4341bbabf552fe028b8b` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:19.076133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb64038a84d4341bbabf552fe028b8b)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:19.078441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:19.303757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb64038a84d4341bbabf552fe028b8b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:19.317426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb64038a84d4341bbabf552fe028b8b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:19.324669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb64038a84d4341bbabf552fe028b8b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:19.325417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb64038a84d4341bbabf552fe028b8b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:19.326097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3f2eb878-933b-485e-a82c-62d9eb7553a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb64038a84d4341bbabf552fe028b8b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:19.416628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 939/1846] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:18 2022] DELETE /identity/v3/users/abb64038a84d4341bbabf552fe028b8b => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:19.546229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:20.017987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:20.021077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:20.057527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bcae8-a0be-420c-b331-89153119d783 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:20.183153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:20.185422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:20.188398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:20.188398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] PATH_INFO: `/v3/projects/07622697f04e4cebb8aa3a233c62d609` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:20.188398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07622697f04e4cebb8aa3a233c62d609)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:20.193950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:20.920899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07622697f04e4cebb8aa3a233c62d609', 'request_id': 'req-49714167-d197-45e8-ae16-2fb84a9ee4c5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:21.171840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07622697f04e4cebb8aa3a233c62d609', 'request_id': 'req-49714167-d197-45e8-ae16-2fb84a9ee4c5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:21.177482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07622697f04e4cebb8aa3a233c62d609', 'request_id': 'req-49714167-d197-45e8-ae16-2fb84a9ee4c5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:21.191563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] The token cache is being invalidate because project 07622697f04e4cebb8aa3a233c62d609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:21.191839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:21.192443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-49714167-d197-45e8-ae16-2fb84a9ee4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:21.194238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 908/1847] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:20 2022] DELETE /identity/v3/projects/07622697f04e4cebb8aa3a233c62d609 => generated 0 bytes in 1177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:21.200362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d53f56-1191-4389-b5ee-1297a38f3642 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.200904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d53f56-1191-4389-b5ee-1297a38f3642 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.201007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d53f56-1191-4389-b5ee-1297a38f3642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.222585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-63d53f56-1191-4389-b5ee-1297a38f3642 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:21.223876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-63d53f56-1191-4389-b5ee-1297a38f3642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:21.326516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 940/1848] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:21.340036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a8ca5-29cb-4386-bfd9-fc935cdc173f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.340445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a8ca5-29cb-4386-bfd9-fc935cdc173f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.340831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a8ca5-29cb-4386-bfd9-fc935cdc173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.350342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45025d-c1ba-4f8d-b1d0-1d4a374eb972 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.350628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45025d-c1ba-4f8d-b1d0-1d4a374eb972 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.350911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45025d-c1ba-4f8d-b1d0-1d4a374eb972 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.352276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 941/1849] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:35:21.358267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e916f2-706b-4eb4-8bd1-c1a2ada2129e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.358621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e916f2-706b-4eb4-8bd1-c1a2ada2129e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.363877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e916f2-706b-4eb4-8bd1-c1a2ada2129e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.363993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-aa9a8ca5-29cb-4386-bfd9-fc935cdc173f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:21.363993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-aa9a8ca5-29cb-4386-bfd9-fc935cdc173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:21.370362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 909/1850] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:21.374632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a8ca5-29cb-4386-bfd9-fc935cdc173f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:21.377669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-89e916f2-706b-4eb4-8bd1-c1a2ada2129e None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:21.378850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-89e916f2-706b-4eb4-8bd1-c1a2ada2129e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:21.385242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a8ca5-29cb-4386-bfd9-fc935cdc173f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:21.390208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:21.390857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.391140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.391416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.392139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:21.392844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:21.394925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:21.409791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 910/1851] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:35:21.414184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:21.426195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e6e0-1de1-4279-9926-0eaed0e7eb12 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:21.431943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f63d42-e1f9-4fee-92ca-3df92db8ea03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:21.432650 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f63d42-e1f9-4fee-92ca-3df92db8ea03 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.432983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f63d42-e1f9-4fee-92ca-3df92db8ea03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.433267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f63d42-e1f9-4fee-92ca-3df92db8ea03 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.433965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f63d42-e1f9-4fee-92ca-3df92db8ea03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:21.435869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f63d42-e1f9-4fee-92ca-3df92db8ea03 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:21.478124 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 942/1852] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:35:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:21.497618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e916f2-706b-4eb4-8bd1-c1a2ada2129e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:21.501877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-89e916f2-706b-4eb4-8bd1-c1a2ada2129e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:21.515472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e916f2-706b-4eb4-8bd1-c1a2ada2129e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:21.521834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23de0-a34a-42e0-b1a1-836f95b353e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:21.522502 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23de0-a34a-42e0-b1a1-836f95b353e0 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.522833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23de0-a34a-42e0-b1a1-836f95b353e0 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.523121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23de0-a34a-42e0-b1a1-836f95b353e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.532908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e23de0-a34a-42e0-b1a1-836f95b353e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:21.535577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e23de0-a34a-42e0-b1a1-836f95b353e0 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:21.546732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 943/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:21.656756 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 911/1854] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:21 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:21.661347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23de0-a34a-42e0-b1a1-836f95b353e0 service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:21.672506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23de0-a34a-42e0-b1a1-836f95b353e0 service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:21.805054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82c21e-8a86-4f7e-a724-1fc33a0a552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:21.808038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82c21e-8a86-4f7e-a724-1fc33a0a552a admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.808354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82c21e-8a86-4f7e-a724-1fc33a0a552a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.808669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82c21e-8a86-4f7e-a724-1fc33a0a552a admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.809415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82c21e-8a86-4f7e-a724-1fc33a0a552a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:21.815977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82c21e-8a86-4f7e-a724-1fc33a0a552a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:21.931114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 944/1855] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:21 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:21.944799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f63d42-e1f9-4fee-92ca-3df92db8ea03 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:21.962272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f63d42-e1f9-4fee-92ca-3df92db8ea03 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:21.968686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe75ee4-c7c2-4c59-a1ce-53fb6064fd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:21.969607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe75ee4-c7c2-4c59-a1ce-53fb6064fd99 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:21.969896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe75ee4-c7c2-4c59-a1ce-53fb6064fd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:21.970181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe75ee4-c7c2-4c59-a1ce-53fb6064fd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:21.970941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe75ee4-c7c2-4c59-a1ce-53fb6064fd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:21.973009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe75ee4-c7c2-4c59-a1ce-53fb6064fd99 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:21.987067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 912/1856] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:21.988232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82c21e-8a86-4f7e-a724-1fc33a0a552a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:21.998404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82c21e-8a86-4f7e-a724-1fc33a0a552a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:22.004219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2ae5b-0d52-40ff-a108-6af1b2e2d1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:22.005203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2ae5b-0d52-40ff-a108-6af1b2e2d1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.005520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2ae5b-0d52-40ff-a108-6af1b2e2d1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.005811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2ae5b-0d52-40ff-a108-6af1b2e2d1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:22.006556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2ae5b-0d52-40ff-a108-6af1b2e2d1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:22.008173 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2ae5b-0d52-40ff-a108-6af1b2e2d1c4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:22.025455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 945/1857] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:22.029752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe75ee4-c7c2-4c59-a1ce-53fb6064fd99 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:22.042189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe75ee4-c7c2-4c59-a1ce-53fb6064fd99 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:22.048739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be97dc3-76a5-4aa8-a229-5fe15b5e3296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:22.048739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97dc3-76a5-4aa8-a229-5fe15b5e3296 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.048926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97dc3-76a5-4aa8-a229-5fe15b5e3296 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.049120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97dc3-76a5-4aa8-a229-5fe15b5e3296 admin admin] PATH_INFO: `/v3/projects/3fe592427aa747a9ad91075befeb7191/users/4cc935bb96504178abeff2f1c870c147/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:22.066637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be97dc3-76a5-4aa8-a229-5fe15b5e3296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe592427aa747a9ad91075befeb7191, user_id=4cc935bb96504178abeff2f1c870c147, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:22.071734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be97dc3-76a5-4aa8-a229-5fe15b5e3296 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:22.147147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 913/1858] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:22 2022] PUT /identity/v3/projects/3fe592427aa747a9ad91075befeb7191/users/4cc935bb96504178abeff2f1c870c147/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:22.158434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def76f-0ee1-4eb2-8dd1-585a68b86764 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.158728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def76f-0ee1-4eb2-8dd1-585a68b86764 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.159005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def76f-0ee1-4eb2-8dd1-585a68b86764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:22.190651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-54def76f-0ee1-4eb2-8dd1-585a68b86764 None None] MFA Rules not processed for user `4cc935bb96504178abeff2f1c870c147`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:22.191861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-54def76f-0ee1-4eb2-8dd1-585a68b86764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:22.268558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 946/1859] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:22.273090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be97dc3-76a5-4aa8-a229-5fe15b5e3296 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:22.337427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be97dc3-76a5-4aa8-a229-5fe15b5e3296 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:22.343214 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf407ce-047f-44d1-ab6e-beb8896fac00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:22.344029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf407ce-047f-44d1-ab6e-beb8896fac00 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.344478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf407ce-047f-44d1-ab6e-beb8896fac00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.344953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf407ce-047f-44d1-ab6e-beb8896fac00 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:22.346024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf407ce-047f-44d1-ab6e-beb8896fac00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:22.348583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf407ce-047f-44d1-ab6e-beb8896fac00 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:22.435337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 914/1860] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:22 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:22.436884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54def76f-0ee1-4eb2-8dd1-585a68b86764 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:22.452069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54def76f-0ee1-4eb2-8dd1-585a68b86764 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:22.552112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50754f5b-786b-4942-b21d-b9f199bf85e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:22.553479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50754f5b-786b-4942-b21d-b9f199bf85e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.553761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50754f5b-786b-4942-b21d-b9f199bf85e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.554044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50754f5b-786b-4942-b21d-b9f199bf85e4 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:22.555520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50754f5b-786b-4942-b21d-b9f199bf85e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:22.557583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50754f5b-786b-4942-b21d-b9f199bf85e4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:22.720576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:22.728726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 947/1861] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:22 2022] POST /identity/v3/users => generated 290 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:22.729769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf407ce-047f-44d1-ab6e-beb8896fac00 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:22.742275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf407ce-047f-44d1-ab6e-beb8896fac00 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:22.747833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fedb8-75a0-4055-84b1-d44afe02e6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:22.748753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fedb8-75a0-4055-84b1-d44afe02e6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.750915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fedb8-75a0-4055-84b1-d44afe02e6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.750915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fedb8-75a0-4055-84b1-d44afe02e6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:22.750915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fedb8-75a0-4055-84b1-d44afe02e6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:22.754050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fedb8-75a0-4055-84b1-d44afe02e6e1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:22.773304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50754f5b-786b-4942-b21d-b9f199bf85e4 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:22.780004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 915/1862] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:22.789445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50754f5b-786b-4942-b21d-b9f199bf85e4 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:22.795451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca1c20-a729-41c4-bd7c-07cb2d9a0dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:22.798424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1c20-a729-41c4-bd7c-07cb2d9a0dbe admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.798424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1c20-a729-41c4-bd7c-07cb2d9a0dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.798424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1c20-a729-41c4-bd7c-07cb2d9a0dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:22.798424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca1c20-a729-41c4-bd7c-07cb2d9a0dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:22.799321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca1c20-a729-41c4-bd7c-07cb2d9a0dbe admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:22.811435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 948/1863] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:22.817721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fedb8-75a0-4055-84b1-d44afe02e6e1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:22.829674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fedb8-75a0-4055-84b1-d44afe02e6e1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:22.835276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6cf03-e904-487b-b964-e73335dc50e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:22.837066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6cf03-e904-487b-b964-e73335dc50e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.837841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6cf03-e904-487b-b964-e73335dc50e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.838667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6cf03-e904-487b-b964-e73335dc50e0 admin admin] PATH_INFO: `/v3/projects/365c782a11314dcab814f35e8a09d72f/users/0c5dfb198e4146d693c730445ae325ae/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:22.858228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6cf03-e904-487b-b964-e73335dc50e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365c782a11314dcab814f35e8a09d72f, user_id=0c5dfb198e4146d693c730445ae325ae, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:22.860937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6cf03-e904-487b-b964-e73335dc50e0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:22.952158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 916/1864] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:22 2022] PUT /identity/v3/projects/365c782a11314dcab814f35e8a09d72f/users/0c5dfb198e4146d693c730445ae325ae/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:22.963163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd9e1b-f644-4a02-aa27-8ecdfabedff8 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:22.963727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd9e1b-f644-4a02-aa27-8ecdfabedff8 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:22.965389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd9e1b-f644-4a02-aa27-8ecdfabedff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:23.000084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a3fd9e1b-f644-4a02-aa27-8ecdfabedff8 None None] MFA Rules not processed for user `0c5dfb198e4146d693c730445ae325ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:23.002477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a3fd9e1b-f644-4a02-aa27-8ecdfabedff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:23.077867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 949/1865] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:23.078919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6cf03-e904-487b-b964-e73335dc50e0 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:23.173413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6cf03-e904-487b-b964-e73335dc50e0 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:23.179650 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b9fdc-6150-4403-9003-5a702a2050c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:23.183975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9fdc-6150-4403-9003-5a702a2050c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:23.184353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9fdc-6150-4403-9003-5a702a2050c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:23.193686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9fdc-6150-4403-9003-5a702a2050c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:23.194446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b9fdc-6150-4403-9003-5a702a2050c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:23.205799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b9fdc-6150-4403-9003-5a702a2050c5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:23.242786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:23.355390 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 917/1866] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:23 2022] POST /identity/v3/projects => generated 379 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:23.357694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd9e1b-f644-4a02-aa27-8ecdfabedff8 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:23.369147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd9e1b-f644-4a02-aa27-8ecdfabedff8 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:23.427191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a01d91-5c4d-41d1-9910-1f77f3c6a3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:23.427922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a01d91-5c4d-41d1-9910-1f77f3c6a3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:23.428301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a01d91-5c4d-41d1-9910-1f77f3c6a3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:23.428720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a01d91-5c4d-41d1-9910-1f77f3c6a3e8 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:23.429829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a01d91-5c4d-41d1-9910-1f77f3c6a3e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:23.431925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a01d91-5c4d-41d1-9910-1f77f3c6a3e8 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:23.584394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 950/1867] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:23 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:23.588455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b9fdc-6150-4403-9003-5a702a2050c5 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:23.599895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b9fdc-6150-4403-9003-5a702a2050c5 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:23.604953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddf803-55dd-45e8-98f9-0e71a3d9bfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:23.605871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddf803-55dd-45e8-98f9-0e71a3d9bfcd admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:23.606343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddf803-55dd-45e8-98f9-0e71a3d9bfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:23.606839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddf803-55dd-45e8-98f9-0e71a3d9bfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:23.607759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ddf803-55dd-45e8-98f9-0e71a3d9bfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:23.609713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ddf803-55dd-45e8-98f9-0e71a3d9bfcd admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:23.623526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 918/1868] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:23.628483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a01d91-5c4d-41d1-9910-1f77f3c6a3e8 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:23.646298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a01d91-5c4d-41d1-9910-1f77f3c6a3e8 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:23.655454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd7935-f89a-43ed-8ed4-fb0ed7f237f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:23.655454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd7935-f89a-43ed-8ed4-fb0ed7f237f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:23.655454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd7935-f89a-43ed-8ed4-fb0ed7f237f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:23.655454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd7935-f89a-43ed-8ed4-fb0ed7f237f9 admin admin] PATH_INFO: `/v3/projects/4e55f96e79c84755838722bb68907cd3/users/227a1fa6c866470eaca068ac38c0ce5b/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:23.669714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd7935-f89a-43ed-8ed4-fb0ed7f237f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e55f96e79c84755838722bb68907cd3, user_id=227a1fa6c866470eaca068ac38c0ce5b, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:23.672792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd7935-f89a-43ed-8ed4-fb0ed7f237f9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:23.811179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 951/1869] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:23 2022] PUT /identity/v3/projects/4e55f96e79c84755838722bb68907cd3/users/227a1fa6c866470eaca068ac38c0ce5b/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:23.813883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddf803-55dd-45e8-98f9-0e71a3d9bfcd admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:23.824759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddf803-55dd-45e8-98f9-0e71a3d9bfcd admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:23.943536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cf451-8163-4322-a382-a55b62dfc8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:23.944495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf451-8163-4322-a382-a55b62dfc8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:23.945015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf451-8163-4322-a382-a55b62dfc8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:23.945446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf451-8163-4322-a382-a55b62dfc8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:23.953069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cf451-8163-4322-a382-a55b62dfc8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:23.956131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cf451-8163-4322-a382-a55b62dfc8f4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:23.971803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 919/1870] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:23.975216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd7935-f89a-43ed-8ed4-fb0ed7f237f9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:23.986740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd7935-f89a-43ed-8ed4-fb0ed7f237f9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:23.991665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:23.992589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:23.992911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:23.993199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] PATH_INFO: `/v3/projects/4e55f96e79c84755838722bb68907cd3/users/227a1fa6c866470eaca068ac38c0ce5b/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:23.995323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e55f96e79c84755838722bb68907cd3, user_id=227a1fa6c866470eaca068ac38c0ce5b, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:23.998209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:24.010395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 952/1871] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:23 2022] PUT /identity/v3/projects/4e55f96e79c84755838722bb68907cd3/users/227a1fa6c866470eaca068ac38c0ce5b/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:24.022667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b8ae-2f6d-4c3b-bb3d-8f8e2bef70f1 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:24.022962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b8ae-2f6d-4c3b-bb3d-8f8e2bef70f1 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:24.023246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b8ae-2f6d-4c3b-bb3d-8f8e2bef70f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:24.050792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e161b8ae-2f6d-4c3b-bb3d-8f8e2bef70f1 None None] MFA Rules not processed for user `227a1fa6c866470eaca068ac38c0ce5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:24.052089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e161b8ae-2f6d-4c3b-bb3d-8f8e2bef70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:24.126146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 920/1872] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:24.143281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:24.145114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:24.205765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803723fa-36bd-40e1-ae92-20636a6b4888 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:24.209407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:24.210056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:24.210339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:24.210617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:24.219490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:24.220998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:24.230335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 953/1873] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:24.429977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:24.431301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161b8ae-2f6d-4c3b-bb3d-8f8e2bef70f1 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:24.431749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:24.433397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e161b8ae-2f6d-4c3b-bb3d-8f8e2bef70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:24.447556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491e26c-afa0-454c-986b-111cfcae47ae service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:24.452653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:24.453747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:24.454416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:24.455518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:24.509762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161b8ae-2f6d-4c3b-bb3d-8f8e2bef70f1 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:24.514612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:24.515488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:24.515948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:24.516237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] PATH_INFO: `/v3/users/8be5459f856044ce9d40fb75aa9dc8af` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:24.517537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be5459f856044ce9d40fb75aa9dc8af)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:24.519461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:24.522044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:24.523675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:24.534161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 954/1874] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:24.802224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:24.803727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:24.820102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be27e5b-1f93-489a-ad95-4283c8e550fa service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:24.824730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:24.825893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:24.825977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:24.826312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] PATH_INFO: `/v3/users/08c1dc4c35d146b383af36fb271800d1` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:24.827568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c1dc4c35d146b383af36fb271800d1)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:24.829687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:24.903396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be5459f856044ce9d40fb75aa9dc8af'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:24.911515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be5459f856044ce9d40fb75aa9dc8af'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:24.922290 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be5459f856044ce9d40fb75aa9dc8af'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:24.923088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be5459f856044ce9d40fb75aa9dc8af'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:24.974654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:25.034993 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bebe2419-c63f-4ab5-b8e1-abc9a633fe65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be5459f856044ce9d40fb75aa9dc8af'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:25.053489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 921/1875] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:24 2022] DELETE /identity/v3/users/8be5459f856044ce9d40fb75aa9dc8af => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:25.057862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466bdab-c3fe-4136-ad25-d3ef0d9e2b59 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:25.058217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466bdab-c3fe-4136-ad25-d3ef0d9e2b59 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:25.058503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466bdab-c3fe-4136-ad25-d3ef0d9e2b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:25.084579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a466bdab-c3fe-4136-ad25-d3ef0d9e2b59 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:25.086126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a466bdab-c3fe-4136-ad25-d3ef0d9e2b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:25.106098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c1dc4c35d146b383af36fb271800d1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:25.137037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c1dc4c35d146b383af36fb271800d1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:25.157168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c1dc4c35d146b383af36fb271800d1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:25.160011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c1dc4c35d146b383af36fb271800d1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:25.160752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8499c85b-76e4-473f-994e-7ffc4cd9b1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c1dc4c35d146b383af36fb271800d1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:25.182144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 922/1876] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:25.192306 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466bdab-c3fe-4136-ad25-d3ef0d9e2b59 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:25.193885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a466bdab-c3fe-4136-ad25-d3ef0d9e2b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:25.257102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466bdab-c3fe-4136-ad25-d3ef0d9e2b59 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:25.260856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd58cf-e2ec-49f0-b19b-718ab16300ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:25.261517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd58cf-e2ec-49f0-b19b-718ab16300ec service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:25.261805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd58cf-e2ec-49f0-b19b-718ab16300ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:25.262090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd58cf-e2ec-49f0-b19b-718ab16300ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:25.271917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd58cf-e2ec-49f0-b19b-718ab16300ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:25.273541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd58cf-e2ec-49f0-b19b-718ab16300ec service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:25.276934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 955/1877] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:24 2022] DELETE /identity/v3/users/08c1dc4c35d146b383af36fb271800d1 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:25.281325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef136f-fb09-46f4-9eba-28bf899fc31b None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:25.281621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef136f-fb09-46f4-9eba-28bf899fc31b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:25.281907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef136f-fb09-46f4-9eba-28bf899fc31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:25.284574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 923/1878] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:25.320804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-3eef136f-fb09-46f4-9eba-28bf899fc31b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:25.323667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3eef136f-fb09-46f4-9eba-28bf899fc31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:25.397474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 956/1879] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:25.406933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd58cf-e2ec-49f0-b19b-718ab16300ec service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:25.416380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd58cf-e2ec-49f0-b19b-718ab16300ec service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:25.471765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:25.472467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:25.472784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:25.473893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:25.485417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:25.487126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:25.496827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 924/1880] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:25.820665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef136f-fb09-46f4-9eba-28bf899fc31b None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:25.822679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3eef136f-fb09-46f4-9eba-28bf899fc31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:25.837976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef136f-fb09-46f4-9eba-28bf899fc31b None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:25.843153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:25.844652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:25.844985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:25.846518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] PATH_INFO: `/v3/projects/577c1a867889439782a4154e52064520` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:25.848889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=577c1a867889439782a4154e52064520)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:25.851080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:25.973940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '577c1a867889439782a4154e52064520', 'request_id': 'req-976fa921-9812-4094-924b-e101199955a0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.054885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '577c1a867889439782a4154e52064520', 'request_id': 'req-976fa921-9812-4094-924b-e101199955a0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.065137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:26.066012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '577c1a867889439782a4154e52064520', 'request_id': 'req-976fa921-9812-4094-924b-e101199955a0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.067073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:26.083804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] The token cache is being invalidate because project 577c1a867889439782a4154e52064520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:26.084102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.084845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.089267 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 957/1881] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:25 2022] DELETE /identity/v3/projects/577c1a867889439782a4154e52064520 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:26.093804 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef590ca-b9b9-48de-a1b6-86213c95119e service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:26.096290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:26.099973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:26.174994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:26.176027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:26.176358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:26.178997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] PATH_INFO: `/v3/projects/1604f4a9e69d4667a255e02b0de2e538` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:26.178997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1604f4a9e69d4667a255e02b0de2e538)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:26.182356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa921-9812-4094-924b-e101199955a0 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:26.182790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:26.188204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:26.189790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:26.190176 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:26.190556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] PATH_INFO: `/v3/users/8efcb2a11c784cdfa3957703eeb893bb` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:26.192656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8efcb2a11c784cdfa3957703eeb893bb)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:26.195333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:26.422683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:26.440560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8efcb2a11c784cdfa3957703eeb893bb'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.463043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1604f4a9e69d4667a255e02b0de2e538', 'request_id': 'req-c39b0c66-39fc-4181-85c2-7815d2e3c689'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.468679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8efcb2a11c784cdfa3957703eeb893bb'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.480442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8efcb2a11c784cdfa3957703eeb893bb'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.480442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8efcb2a11c784cdfa3957703eeb893bb'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.480442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8efcb2a11c784cdfa3957703eeb893bb'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.579374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1604f4a9e69d4667a255e02b0de2e538', 'request_id': 'req-c39b0c66-39fc-4181-85c2-7815d2e3c689'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.591919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1604f4a9e69d4667a255e02b0de2e538', 'request_id': 'req-c39b0c66-39fc-4181-85c2-7815d2e3c689'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.610472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] The token cache is being invalidate because project 1604f4a9e69d4667a255e02b0de2e538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:26.611066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.612732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:26.615301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 925/1882] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:26 2022] DELETE /identity/v3/projects/1604f4a9e69d4667a255e02b0de2e538 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:26.621898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:26.624242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:26.669297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 958/1883] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:26 2022] DELETE /identity/v3/users/8efcb2a11c784cdfa3957703eeb893bb => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:26.737584 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b0c66-39fc-4181-85c2-7815d2e3c689 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:26.839173 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:26.840377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:26.840699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:26.840987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] PATH_INFO: `/v3/users/2bc26ecfd13547319c2ef23fc0aa8c22` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:26.845006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc26ecfd13547319c2ef23fc0aa8c22)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:26.848920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:26.915577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:27.148321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc26ecfd13547319c2ef23fc0aa8c22'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.159037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc26ecfd13547319c2ef23fc0aa8c22'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.168807 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc26ecfd13547319c2ef23fc0aa8c22'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.170030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc26ecfd13547319c2ef23fc0aa8c22'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.253074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-162560a6-d822-4777-8e3d-79fffd63c02b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc26ecfd13547319c2ef23fc0aa8c22'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.293328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 926/1884] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:26 2022] DELETE /identity/v3/users/2bc26ecfd13547319c2ef23fc0aa8c22 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:27.390382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:27.393684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:27.422581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e5bc9-77d3-4823-a7a5-d280be2de3b3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:27.560429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:27.561463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:27.561823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:27.562184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] PATH_INFO: `/v3/projects/8101fcbce6304257bbb63661d2b6a1e4` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:27.563479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8101fcbce6304257bbb63661d2b6a1e4)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:27.566501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:27.737856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8101fcbce6304257bbb63661d2b6a1e4', 'request_id': 'req-be87b1c9-edb2-415a-9699-5aaf28b7105b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.831332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8101fcbce6304257bbb63661d2b6a1e4', 'request_id': 'req-be87b1c9-edb2-415a-9699-5aaf28b7105b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.837778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8101fcbce6304257bbb63661d2b6a1e4', 'request_id': 'req-be87b1c9-edb2-415a-9699-5aaf28b7105b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.850441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] The token cache is being invalidate because project 8101fcbce6304257bbb63661d2b6a1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:27.850725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.851378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:27.853014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 959/1885] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:27 2022] DELETE /identity/v3/projects/8101fcbce6304257bbb63661d2b6a1e4 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:27.858141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa0e1-28ef-415a-b406-8ae8e5ba5605 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:27.858664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa0e1-28ef-415a-b406-8ae8e5ba5605 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:27.858889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa0e1-28ef-415a-b406-8ae8e5ba5605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:27.877835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9d3aa0e1-28ef-415a-b406-8ae8e5ba5605 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:27.879309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9d3aa0e1-28ef-415a-b406-8ae8e5ba5605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:27.887545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:27.890321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:27.971062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 927/1886] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:27.973381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64538ce-7ff3-41be-b942-d62138344709 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:27.973867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64538ce-7ff3-41be-b942-d62138344709 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:27.974602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64538ce-7ff3-41be-b942-d62138344709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.002735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d64538ce-7ff3-41be-b942-d62138344709 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:28.004160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d64538ce-7ff3-41be-b942-d62138344709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:28.017225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 928/1887] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:28.023639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87b1c9-edb2-415a-9699-5aaf28b7105b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.023998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64538ce-7ff3-41be-b942-d62138344709 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.036272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.037348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.037589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.037983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] PATH_INFO: `/v3/projects/8510c11100914ad9b872c6d31a39f5ab` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.039630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=8510c11100914ad9b872c6d31a39f5ab)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.042051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.046828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64538ce-7ff3-41be-b942-d62138344709 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.055547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.056328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.056742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.057113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.057929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:28.058314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.060060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.073436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 929/1888] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:28.078094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.088262 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ca860-d763-4440-b793-09bdc7088403 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.093120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abdc5-3f30-42c8-ab5e-6274b79b4177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.093798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abdc5-3f30-42c8-ab5e-6274b79b4177 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.094090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abdc5-3f30-42c8-ab5e-6274b79b4177 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.094374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abdc5-3f30-42c8-ab5e-6274b79b4177 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.095811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866abdc5-3f30-42c8-ab5e-6274b79b4177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.097694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866abdc5-3f30-42c8-ab5e-6274b79b4177 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.198093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8510c11100914ad9b872c6d31a39f5ab', 'request_id': 'req-9efe47d9-2ed6-44c9-a379-b189d1b972ec'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:28.217875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 930/1889] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:28.222828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abdc5-3f30-42c8-ab5e-6274b79b4177 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.285147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abdc5-3f30-42c8-ab5e-6274b79b4177 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.289486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8510c11100914ad9b872c6d31a39f5ab', 'request_id': 'req-9efe47d9-2ed6-44c9-a379-b189d1b972ec'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:28.290149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.290871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.292355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.295707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.295707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.295707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.296680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8510c11100914ad9b872c6d31a39f5ab', 'request_id': 'req-9efe47d9-2ed6-44c9-a379-b189d1b972ec'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:28.308283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] The token cache is being invalidate because project 8510c11100914ad9b872c6d31a39f5ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:28.308602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:28.309299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9efe47d9-2ed6-44c9-a379-b189d1b972ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:28.310857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 960/1890] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:27 2022] DELETE /identity/v3/projects/8510c11100914ad9b872c6d31a39f5ab => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:28.316316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002e95-1cad-4ecc-89db-809ab4cda5ab None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.316726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002e95-1cad-4ecc-89db-809ab4cda5ab None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.317076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002e95-1cad-4ecc-89db-809ab4cda5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.336365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-8a002e95-1cad-4ecc-89db-809ab4cda5ab None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:28.337835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8a002e95-1cad-4ecc-89db-809ab4cda5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:28.353908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 961/1891] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:28.356080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b225ca-c59b-4be7-884f-d49df739385d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.356426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b225ca-c59b-4be7-884f-d49df739385d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.356741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b225ca-c59b-4be7-884f-d49df739385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.377833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-27b225ca-c59b-4be7-884f-d49df739385d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:28.379074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-27b225ca-c59b-4be7-884f-d49df739385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:28.388183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 962/1892] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:28.393100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b225ca-c59b-4be7-884f-d49df739385d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.406032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 931/1893] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:28.408560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b225ca-c59b-4be7-884f-d49df739385d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.413865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.421321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.422070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.422362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.422712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.423572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:28.423920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.424955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:28.425652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.442887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 963/1894] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:28.448677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b9afd-dfa7-4189-a495-a6122bbeb422 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.451439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.455723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a398c-0fae-4eea-afe9-ab254bd65ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.456924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a398c-0fae-4eea-afe9-ab254bd65ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.457422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a398c-0fae-4eea-afe9-ab254bd65ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.462932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a398c-0fae-4eea-afe9-ab254bd65ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.462932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a398c-0fae-4eea-afe9-ab254bd65ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.462932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a398c-0fae-4eea-afe9-ab254bd65ac1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.463427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43a04d-0254-4383-a958-29a7c9c2099e admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.473556 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 932/1895] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:28.477147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a398c-0fae-4eea-afe9-ab254bd65ac1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.479389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008117f-7400-42a4-afd3-858cad79daf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.480156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008117f-7400-42a4-afd3-858cad79daf9 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.480478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008117f-7400-42a4-afd3-858cad79daf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.480807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008117f-7400-42a4-afd3-858cad79daf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.482065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008117f-7400-42a4-afd3-858cad79daf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.483965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008117f-7400-42a4-afd3-858cad79daf9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.491734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a398c-0fae-4eea-afe9-ab254bd65ac1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.497593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94f62-a055-4de8-bfd6-e991f443a07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.498656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94f62-a055-4de8-bfd6-e991f443a07b admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.499182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94f62-a055-4de8-bfd6-e991f443a07b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.499692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94f62-a055-4de8-bfd6-e991f443a07b admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.500684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94f62-a055-4de8-bfd6-e991f443a07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.502555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94f62-a055-4de8-bfd6-e991f443a07b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.518114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 933/1896] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:28.519561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94f62-a055-4de8-bfd6-e991f443a07b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.533464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94f62-a055-4de8-bfd6-e991f443a07b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.538379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce6ee5-0d5a-4e36-9f68-4ac87662f278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.539305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce6ee5-0d5a-4e36-9f68-4ac87662f278 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.539593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce6ee5-0d5a-4e36-9f68-4ac87662f278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.539876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce6ee5-0d5a-4e36-9f68-4ac87662f278 admin admin] PATH_INFO: `/v3/projects/14bca1e13e8845b49e9f6c57f32477dd/users/c495bca3ac334d7ca7cd7f76db6be1a2/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.555491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce6ee5-0d5a-4e36-9f68-4ac87662f278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14bca1e13e8845b49e9f6c57f32477dd, user_id=c495bca3ac334d7ca7cd7f76db6be1a2, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.558783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce6ee5-0d5a-4e36-9f68-4ac87662f278 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.613579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 964/1897] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/projects => generated 365 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:28.614792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008117f-7400-42a4-afd3-858cad79daf9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.661377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 934/1898] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:28 2022] PUT /identity/v3/projects/14bca1e13e8845b49e9f6c57f32477dd/users/c495bca3ac334d7ca7cd7f76db6be1a2/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:28.673234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5a17d-432a-49f4-9425-0230aecaa51d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.673643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5a17d-432a-49f4-9425-0230aecaa51d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.674004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5a17d-432a-49f4-9425-0230aecaa51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.684236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:28.687096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008117f-7400-42a4-afd3-858cad79daf9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.706227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-93a5a17d-432a-49f4-9425-0230aecaa51d None None] MFA Rules not processed for user `c495bca3ac334d7ca7cd7f76db6be1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:28.707558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-93a5a17d-432a-49f4-9425-0230aecaa51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:28.760497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ebb82-31ee-498c-b631-43f2e9de6552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.761349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ebb82-31ee-498c-b631-43f2e9de6552 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.761701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ebb82-31ee-498c-b631-43f2e9de6552 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.762057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ebb82-31ee-498c-b631-43f2e9de6552 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.763200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ebb82-31ee-498c-b631-43f2e9de6552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.765303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ebb82-31ee-498c-b631-43f2e9de6552 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.792953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 935/1899] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:28.799174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5a17d-432a-49f4-9425-0230aecaa51d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.813931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5a17d-432a-49f4-9425-0230aecaa51d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.819259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bd674-2bfa-4336-a463-8c88701b8163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.819933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bd674-2bfa-4336-a463-8c88701b8163 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.820225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bd674-2bfa-4336-a463-8c88701b8163 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.820507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bd674-2bfa-4336-a463-8c88701b8163 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.821357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bd674-2bfa-4336-a463-8c88701b8163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.823354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bd674-2bfa-4336-a463-8c88701b8163 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.872389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 965/1900] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/users => generated 283 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:28.877689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ebb82-31ee-498c-b631-43f2e9de6552 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.890771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ebb82-31ee-498c-b631-43f2e9de6552 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.901717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f792b-5848-4222-8f39-295a6f11cca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.902690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f792b-5848-4222-8f39-295a6f11cca1 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.903983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f792b-5848-4222-8f39-295a6f11cca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.904290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f792b-5848-4222-8f39-295a6f11cca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.905112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f792b-5848-4222-8f39-295a6f11cca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.907067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f792b-5848-4222-8f39-295a6f11cca1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.919102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 966/1901] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:28.922926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f792b-5848-4222-8f39-295a6f11cca1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.934602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f792b-5848-4222-8f39-295a6f11cca1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.940343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dff1ba-64de-40d2-a57d-c4afb82c1799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.941185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dff1ba-64de-40d2-a57d-c4afb82c1799 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.941553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dff1ba-64de-40d2-a57d-c4afb82c1799 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.941838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dff1ba-64de-40d2-a57d-c4afb82c1799 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.942611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dff1ba-64de-40d2-a57d-c4afb82c1799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.944185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dff1ba-64de-40d2-a57d-c4afb82c1799 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.962762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 936/1902] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:28.963812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bd674-2bfa-4336-a463-8c88701b8163 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.969278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 967/1903] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:28.973558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dff1ba-64de-40d2-a57d-c4afb82c1799 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.992638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dff1ba-64de-40d2-a57d-c4afb82c1799 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.075986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:29.082921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bd674-2bfa-4336-a463-8c88701b8163 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.099509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50e072-55fa-4b41-973d-f9585b6e5c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.100561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e072-55fa-4b41-973d-f9585b6e5c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.100866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e072-55fa-4b41-973d-f9585b6e5c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.101149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e072-55fa-4b41-973d-f9585b6e5c3a admin admin] PATH_INFO: `/v3/projects/cdf2e66adadf458da01c96754653686e/users/4473832ed9654d40b4b2d30593e12c13/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.102140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84053a-6a33-4ad8-900b-c871edc9ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.103239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84053a-6a33-4ad8-900b-c871edc9ccf8 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.103660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84053a-6a33-4ad8-900b-c871edc9ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.104198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84053a-6a33-4ad8-900b-c871edc9ccf8 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.105164 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84053a-6a33-4ad8-900b-c871edc9ccf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.109302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84053a-6a33-4ad8-900b-c871edc9ccf8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.124180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50e072-55fa-4b41-973d-f9585b6e5c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf2e66adadf458da01c96754653686e, user_id=4473832ed9654d40b4b2d30593e12c13, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.128413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50e072-55fa-4b41-973d-f9585b6e5c3a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.233375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 968/1904] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:28 2022] PUT /identity/v3/projects/cdf2e66adadf458da01c96754653686e/users/4473832ed9654d40b4b2d30593e12c13/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:29.243976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf1ba6-4a5b-4f0e-912c-e77dcd27aaef None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.244369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf1ba6-4a5b-4f0e-912c-e77dcd27aaef None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.245504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf1ba6-4a5b-4f0e-912c-e77dcd27aaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.247575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 937/1905] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/users => generated 296 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:29.251364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84053a-6a33-4ad8-900b-c871edc9ccf8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.263508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84053a-6a33-4ad8-900b-c871edc9ccf8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.279408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-77cf1ba6-4a5b-4f0e-912c-e77dcd27aaef None None] MFA Rules not processed for user `4473832ed9654d40b4b2d30593e12c13`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:29.280562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-77cf1ba6-4a5b-4f0e-912c-e77dcd27aaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:29.330819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.331504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.331787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.332069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.334111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.335711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.347764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 938/1906] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:29.348624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.353822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:29.355188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 969/1907] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:29.355836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf1ba6-4a5b-4f0e-912c-e77dcd27aaef None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.368023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf1ba6-4a5b-4f0e-912c-e77dcd27aaef None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.371837 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb6ba-2a4f-4777-846c-9e8e14574b8c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.374253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d96752-c589-4ace-8be2-150510d29d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.375043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d96752-c589-4ace-8be2-150510d29d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.375451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d96752-c589-4ace-8be2-150510d29d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.375734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d96752-c589-4ace-8be2-150510d29d10 admin admin] PATH_INFO: `/v3/projects/2f124a0b2c2a4592ab5c907190b27508/users/8931e62de1f2496180e50558abb4e1fc/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.377763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.378639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.379656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.379656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] PATH_INFO: `/v3/users/4cc935bb96504178abeff2f1c870c147` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.381134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc935bb96504178abeff2f1c870c147)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.383067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.394532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d96752-c589-4ace-8be2-150510d29d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f124a0b2c2a4592ab5c907190b27508, user_id=8931e62de1f2496180e50558abb4e1fc, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.397851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d96752-c589-4ace-8be2-150510d29d10 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.478844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 970/1908] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:29 2022] PUT /identity/v3/projects/2f124a0b2c2a4592ab5c907190b27508/users/8931e62de1f2496180e50558abb4e1fc/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:29.479998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d96752-c589-4ace-8be2-150510d29d10 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.576951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d96752-c589-4ace-8be2-150510d29d10 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.582571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2344e12-852b-4a00-8ceb-48dabdd8512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.584548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344e12-852b-4a00-8ceb-48dabdd8512d admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.584548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344e12-852b-4a00-8ceb-48dabdd8512d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.584548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344e12-852b-4a00-8ceb-48dabdd8512d admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.584864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2344e12-852b-4a00-8ceb-48dabdd8512d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.587389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2344e12-852b-4a00-8ceb-48dabdd8512d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.786826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 971/1909] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:29 2022] POST /identity/v3/projects => generated 367 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:29.786826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2344e12-852b-4a00-8ceb-48dabdd8512d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.849181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc935bb96504178abeff2f1c870c147'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:29.860030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc935bb96504178abeff2f1c870c147'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:29.861142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc935bb96504178abeff2f1c870c147'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:29.861866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc935bb96504178abeff2f1c870c147'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:29.912594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2344e12-852b-4a00-8ceb-48dabdd8512d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.920287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.921046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.921330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.921636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.922370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.924794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.936399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 972/1910] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:29.937225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.941404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:29.947675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc935bb96504178abeff2f1c870c147'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:29.957765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0ef2f-52d7-47d5-a657-a2680e1d09fc admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.963616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f52406-966b-484a-8980-5dd4cc733675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.964324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f52406-966b-484a-8980-5dd4cc733675 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.964683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f52406-966b-484a-8980-5dd4cc733675 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.965023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f52406-966b-484a-8980-5dd4cc733675 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.965737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f52406-966b-484a-8980-5dd4cc733675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.967888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f52406-966b-484a-8980-5dd4cc733675 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.973323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 939/1911] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:29 2022] DELETE /identity/v3/users/4cc935bb96504178abeff2f1c870c147 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:29.974545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.976999 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:30.053314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fef9c-b24d-4b8d-b9e0-0f9c949a45e5 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:30.058015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156c0fc-5a8a-44bc-9175-b70ecafdb16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:30.058910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156c0fc-5a8a-44bc-9175-b70ecafdb16b admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.059199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156c0fc-5a8a-44bc-9175-b70ecafdb16b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.059485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156c0fc-5a8a-44bc-9175-b70ecafdb16b admin admin] PATH_INFO: `/v3/projects/2f124a0b2c2a4592ab5c907190b27508/users/8931e62de1f2496180e50558abb4e1fc/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.061575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156c0fc-5a8a-44bc-9175-b70ecafdb16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f124a0b2c2a4592ab5c907190b27508, user_id=8931e62de1f2496180e50558abb4e1fc, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:30.066728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156c0fc-5a8a-44bc-9175-b70ecafdb16b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:30.078660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 940/1912] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:29 2022] PUT /identity/v3/projects/2f124a0b2c2a4592ab5c907190b27508/users/8931e62de1f2496180e50558abb4e1fc/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:30.080012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492874a-341d-4cb6-b759-d6c10334c98e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.080299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492874a-341d-4cb6-b759-d6c10334c98e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.080730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492874a-341d-4cb6-b759-d6c10334c98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.101007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 973/1913] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:29 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:30.102698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd00a-13c6-4222-8047-d2dd8c5d3fdf None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.102786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd00a-13c6-4222-8047-d2dd8c5d3fdf None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.103012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd00a-13c6-4222-8047-d2dd8c5d3fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.120744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7492874a-341d-4cb6-b759-d6c10334c98e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:30.122064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7492874a-341d-4cb6-b759-d6c10334c98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:30.152136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-664bd00a-13c6-4222-8047-d2dd8c5d3fdf None None] MFA Rules not processed for user `8931e62de1f2496180e50558abb4e1fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:30.153460 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-664bd00a-13c6-4222-8047-d2dd8c5d3fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:30.212826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 941/1914] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:30.213930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492874a-341d-4cb6-b759-d6c10334c98e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:30.226407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492874a-341d-4cb6-b759-d6c10334c98e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:30.232742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529ac92-8d5e-47c3-9346-3400931aec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:30.233519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529ac92-8d5e-47c3-9346-3400931aec35 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.233867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529ac92-8d5e-47c3-9346-3400931aec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.234217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529ac92-8d5e-47c3-9346-3400931aec35 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.235282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529ac92-8d5e-47c3-9346-3400931aec35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:30.238956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529ac92-8d5e-47c3-9346-3400931aec35 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:30.252316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 974/1915] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:30.252316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bd00a-13c6-4222-8047-d2dd8c5d3fdf None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:30.256786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 942/1916] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:30.258273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-664bd00a-13c6-4222-8047-d2dd8c5d3fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:30.264338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529ac92-8d5e-47c3-9346-3400931aec35 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:30.303208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529ac92-8d5e-47c3-9346-3400931aec35 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:30.325803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a123fb-0197-4ec8-9d43-ac6bb1cc68e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:30.326760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a123fb-0197-4ec8-9d43-ac6bb1cc68e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.327005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a123fb-0197-4ec8-9d43-ac6bb1cc68e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.327430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a123fb-0197-4ec8-9d43-ac6bb1cc68e7 admin admin] PATH_INFO: `/v3/projects/8b9e046016b246c18add6b4ddc28ece1/users/6e8baac43ea54c36b45e2dc01d0f8f24/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.362942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a123fb-0197-4ec8-9d43-ac6bb1cc68e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9e046016b246c18add6b4ddc28ece1, user_id=6e8baac43ea54c36b45e2dc01d0f8f24, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:30.366141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a123fb-0197-4ec8-9d43-ac6bb1cc68e7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:30.388571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bd00a-13c6-4222-8047-d2dd8c5d3fdf None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:30.393287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:30.394054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.394449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.394804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.405819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:30.408352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:30.422834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 975/1917] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:30.425820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:30.427418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:30.484842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 943/1918] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:30 2022] PUT /identity/v3/projects/8b9e046016b246c18add6b4ddc28ece1/users/6e8baac43ea54c36b45e2dc01d0f8f24/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:30.490086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a123fb-0197-4ec8-9d43-ac6bb1cc68e7 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:30.513042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416cea-eac3-4cd4-9fe4-10d490fc133b service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:30.678014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a123fb-0197-4ec8-9d43-ac6bb1cc68e7 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:30.687480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f393-1255-4626-9d04-531dd73c157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:30.688380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f393-1255-4626-9d04-531dd73c157b admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.689598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f393-1255-4626-9d04-531dd73c157b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.690141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f393-1255-4626-9d04-531dd73c157b admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.690960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6f393-1255-4626-9d04-531dd73c157b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:30.692583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6f393-1255-4626-9d04-531dd73c157b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:30.707484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 944/1919] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:30.709642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f393-1255-4626-9d04-531dd73c157b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:30.723804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73b7da-2701-43c3-b55b-004fe2eb4129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:30.724475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73b7da-2701-43c3-b55b-004fe2eb4129 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.725161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73b7da-2701-43c3-b55b-004fe2eb4129 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.725161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73b7da-2701-43c3-b55b-004fe2eb4129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.751159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f393-1255-4626-9d04-531dd73c157b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:30.762788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46df93-bf4a-4e4d-b75f-5abc4593912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:30.763797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46df93-bf4a-4e4d-b75f-5abc4593912f admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.764779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46df93-bf4a-4e4d-b75f-5abc4593912f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.764779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46df93-bf4a-4e4d-b75f-5abc4593912f admin admin] PATH_INFO: `/v3/projects/8b9e046016b246c18add6b4ddc28ece1/users/6e8baac43ea54c36b45e2dc01d0f8f24/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.769838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46df93-bf4a-4e4d-b75f-5abc4593912f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9e046016b246c18add6b4ddc28ece1, user_id=6e8baac43ea54c36b45e2dc01d0f8f24, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:30.776557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46df93-bf4a-4e4d-b75f-5abc4593912f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:30.793497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:30.808478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 945/1920] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:30 2022] PUT /identity/v3/projects/8b9e046016b246c18add6b4ddc28ece1/users/6e8baac43ea54c36b45e2dc01d0f8f24/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:30.820007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8c503-2f58-419c-8dd8-343045c29d3a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:30.820329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8c503-2f58-419c-8dd8-343045c29d3a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:30.820593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8c503-2f58-419c-8dd8-343045c29d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:30.852765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73b7da-2701-43c3-b55b-004fe2eb4129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:30.854324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73b7da-2701-43c3-b55b-004fe2eb4129 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:30.864505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a9b8c503-2f58-419c-8dd8-343045c29d3a None None] MFA Rules not processed for user `6e8baac43ea54c36b45e2dc01d0f8f24`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:30.866188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a9b8c503-2f58-419c-8dd8-343045c29d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:30.867845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 976/1921] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:30.946381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 946/1922] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:31.073656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73b7da-2701-43c3-b55b-004fe2eb4129 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:31.082872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73b7da-2701-43c3-b55b-004fe2eb4129 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:31.104411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8c503-2f58-419c-8dd8-343045c29d3a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:31.106171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a9b8c503-2f58-419c-8dd8-343045c29d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:31.125767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:31.158204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:31.159024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:31.159313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:31.159598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:31.170156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:31.171940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:31.183710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 977/1923] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:31.201143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8c503-2f58-419c-8dd8-343045c29d3a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:31.206754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:31.209818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:31.209818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:31.209818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] PATH_INFO: `/v3/projects/3fe592427aa747a9ad91075befeb7191` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:31.209818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe592427aa747a9ad91075befeb7191)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:31.211513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:31.325719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:31.328875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:31.341453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8afa17-339c-48b2-8022-d47e3853aa93 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:31.345148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:31.345818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:31.346100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:31.346379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:31.348143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:31.366066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe592427aa747a9ad91075befeb7191', 'request_id': 'req-377b6e60-a4dd-4e43-93c4-a45886fb1033'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.415505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:31.416961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:31.425904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe592427aa747a9ad91075befeb7191', 'request_id': 'req-377b6e60-a4dd-4e43-93c4-a45886fb1033'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.427083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 978/1924] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:31.435628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe592427aa747a9ad91075befeb7191', 'request_id': 'req-377b6e60-a4dd-4e43-93c4-a45886fb1033'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.447782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] The token cache is being invalidate because project 3fe592427aa747a9ad91075befeb7191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:31.448257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.449119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.455559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:31.457728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 947/1925] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:31 2022] DELETE /identity/v3/projects/3fe592427aa747a9ad91075befeb7191 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:31.470858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfb380-d7cd-4676-b7e1-f2e50f2689bd service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:31.536678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:31.538132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:31.538792 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:31.539253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] PATH_INFO: `/v3/users/0c5dfb198e4146d693c730445ae325ae` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:31.540805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5dfb198e4146d693c730445ae325ae)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:31.542775 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:31.737655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:31.739344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:31.748974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:31.755655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b6e60-a4dd-4e43-93c4-a45886fb1033 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:31.761563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:31.762509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:31.762717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:31.763129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] PATH_INFO: `/v3/users/c495bca3ac334d7ca7cd7f76db6be1a2` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:31.763619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5dfb198e4146d693c730445ae325ae'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.764470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c495bca3ac334d7ca7cd7f76db6be1a2)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:31.766528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:31.776746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5dfb198e4146d693c730445ae325ae'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.786675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5dfb198e4146d693c730445ae325ae'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.787555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5dfb198e4146d693c730445ae325ae'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.788159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6c43199d-756b-47d5-a9fb-75e6b0eb2532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5dfb198e4146d693c730445ae325ae'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.863199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 979/1926] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:31 2022] DELETE /identity/v3/users/0c5dfb198e4146d693c730445ae325ae => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:31.954335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c495bca3ac334d7ca7cd7f76db6be1a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.961430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c495bca3ac334d7ca7cd7f76db6be1a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.962113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c495bca3ac334d7ca7cd7f76db6be1a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.962736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c495bca3ac334d7ca7cd7f76db6be1a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:31.991696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:32.049208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c495bca3ac334d7ca7cd7f76db6be1a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.089934 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 948/1927] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:31 2022] DELETE /identity/v3/users/c495bca3ac334d7ca7cd7f76db6be1a2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:32.091820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4dbf3-4162-4c55-b1fd-f40b1b11d57c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.092288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4dbf3-4162-4c55-b1fd-f40b1b11d57c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.092690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4dbf3-4162-4c55-b1fd-f40b1b11d57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.120710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-faa4dbf3-4162-4c55-b1fd-f40b1b11d57c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:32.122666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-faa4dbf3-4162-4c55-b1fd-f40b1b11d57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.219381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 980/1928] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:32.230393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.232359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.295198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c5990-3b45-487a-9cf8-2f9dbc480763 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.299321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697b73-3eb0-4edd-b6ec-0142dbb45bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.299969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697b73-3eb0-4edd-b6ec-0142dbb45bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.300254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697b73-3eb0-4edd-b6ec-0142dbb45bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.300544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697b73-3eb0-4edd-b6ec-0142dbb45bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.309497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40697b73-3eb0-4edd-b6ec-0142dbb45bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.310918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40697b73-3eb0-4edd-b6ec-0142dbb45bd3 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.321478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 949/1929] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:32.401166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4dbf3-4162-4c55-b1fd-f40b1b11d57c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.402768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-faa4dbf3-4162-4c55-b1fd-f40b1b11d57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.422283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4dbf3-4162-4c55-b1fd-f40b1b11d57c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.428642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.429666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.446821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.446821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] PATH_INFO: `/v3/projects/365c782a11314dcab814f35e8a09d72f` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.446821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365c782a11314dcab814f35e8a09d72f)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.446821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.571907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365c782a11314dcab814f35e8a09d72f', 'request_id': 'req-ea95ed18-3cdc-47c0-8b52-22e894e09140'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.713920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365c782a11314dcab814f35e8a09d72f', 'request_id': 'req-ea95ed18-3cdc-47c0-8b52-22e894e09140'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.719864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365c782a11314dcab814f35e8a09d72f', 'request_id': 'req-ea95ed18-3cdc-47c0-8b52-22e894e09140'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.731250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] The token cache is being invalidate because project 365c782a11314dcab814f35e8a09d72f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:32.731537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.732185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.733773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 981/1930] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:32 2022] DELETE /identity/v3/projects/365c782a11314dcab814f35e8a09d72f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:32.738537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697b73-3eb0-4edd-b6ec-0142dbb45bd3 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.748250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697b73-3eb0-4edd-b6ec-0142dbb45bd3 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.803050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.803971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.804478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.809313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] PATH_INFO: `/v3/users/227a1fa6c866470eaca068ac38c0ce5b` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.809313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:32.809313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=227a1fa6c866470eaca068ac38c0ce5b)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.809638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.858256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.859652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.872354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95ed18-3cdc-47c0-8b52-22e894e09140 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.877169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.878030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.878317 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.878597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] PATH_INFO: `/v3/projects/14bca1e13e8845b49e9f6c57f32477dd` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.879772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=14bca1e13e8845b49e9f6c57f32477dd)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.886578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:33.002442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '227a1fa6c866470eaca068ac38c0ce5b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.012348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '227a1fa6c866470eaca068ac38c0ce5b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.013729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '227a1fa6c866470eaca068ac38c0ce5b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.015277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '227a1fa6c866470eaca068ac38c0ce5b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.020555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14bca1e13e8845b49e9f6c57f32477dd', 'request_id': 'req-f1371ff9-e5be-46e5-8588-5a8b8496df1f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.080586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:33.108275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '227a1fa6c866470eaca068ac38c0ce5b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.126052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 950/1931] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:32 2022] DELETE /identity/v3/users/227a1fa6c866470eaca068ac38c0ce5b => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:33.178690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14bca1e13e8845b49e9f6c57f32477dd', 'request_id': 'req-f1371ff9-e5be-46e5-8588-5a8b8496df1f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.187367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14bca1e13e8845b49e9f6c57f32477dd', 'request_id': 'req-f1371ff9-e5be-46e5-8588-5a8b8496df1f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.222000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] The token cache is being invalidate because project 14bca1e13e8845b49e9f6c57f32477dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:33.222735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.223736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.228687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 982/1932] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:32 2022] DELETE /identity/v3/projects/14bca1e13e8845b49e9f6c57f32477dd => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:33.232942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:33.234928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:33.323498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd60d-89ed-4ded-80ed-ae79a5200a15 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:33.329279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:33.330395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:33.330686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:33.331029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] PATH_INFO: `/v3/users/8931e62de1f2496180e50558abb4e1fc` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:33.332827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8931e62de1f2496180e50558abb4e1fc)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:33.334794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:33.516134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8931e62de1f2496180e50558abb4e1fc'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.523353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8931e62de1f2496180e50558abb4e1fc'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.524070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8931e62de1f2496180e50558abb4e1fc'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.524755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8931e62de1f2496180e50558abb4e1fc'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.630281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-63b76991-8015-41d0-b345-d7e7591702fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8931e62de1f2496180e50558abb4e1fc'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:33.636606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:33.638188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:33.654559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 951/1933] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:33 2022] DELETE /identity/v3/users/8931e62de1f2496180e50558abb4e1fc => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:33.713421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371ff9-e5be-46e5-8588-5a8b8496df1f admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:33.773425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:33.774321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:33.774624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:33.774908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] PATH_INFO: `/v3/projects/4e55f96e79c84755838722bb68907cd3` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:33.776092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e55f96e79c84755838722bb68907cd3)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:33.778030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:33.919260 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e55f96e79c84755838722bb68907cd3', 'request_id': 'req-6ca12fd4-226a-4a71-8695-beea78650793'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.042425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e55f96e79c84755838722bb68907cd3', 'request_id': 'req-6ca12fd4-226a-4a71-8695-beea78650793'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.048471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e55f96e79c84755838722bb68907cd3', 'request_id': 'req-6ca12fd4-226a-4a71-8695-beea78650793'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.060657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] The token cache is being invalidate because project 4e55f96e79c84755838722bb68907cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:34.060943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.061582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.063346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 983/1934] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:33 2022] DELETE /identity/v3/projects/4e55f96e79c84755838722bb68907cd3 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:34.069242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bba1-696d-4156-8322-ccee8da866b7 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.074975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bba1-696d-4156-8322-ccee8da866b7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.074975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bba1-696d-4156-8322-ccee8da866b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.088115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a720bba1-696d-4156-8322-ccee8da866b7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:34.089346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a720bba1-696d-4156-8322-ccee8da866b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:34.115389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.115938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:34.194199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 952/1935] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:34.202784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d684496-b29e-4c38-a20b-90b82f9e7480 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.203075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d684496-b29e-4c38-a20b-90b82f9e7480 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.203396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d684496-b29e-4c38-a20b-90b82f9e7480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.206720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca12fd4-226a-4a71-8695-beea78650793 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.211099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.212011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.212292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.212621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] PATH_INFO: `/v3/projects/2f124a0b2c2a4592ab5c907190b27508` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.213815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f124a0b2c2a4592ab5c907190b27508)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.215738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.226994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-0d684496-b29e-4c38-a20b-90b82f9e7480 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:34.228273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-0d684496-b29e-4c38-a20b-90b82f9e7480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:34.238961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 953/1936] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:34.243431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d684496-b29e-4c38-a20b-90b82f9e7480 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.253443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d684496-b29e-4c38-a20b-90b82f9e7480 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.259203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.267433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.267745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.268034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.269410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:34.269720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.271321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.286013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 954/1937] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:34.294435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.305394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352150e0-b4ef-4298-8e7e-9e418e66abfd admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.310528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb768-ff01-4727-9567-41e209f23f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.311251 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb768-ff01-4727-9567-41e209f23f73 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.311554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb768-ff01-4727-9567-41e209f23f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.311826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb768-ff01-4727-9567-41e209f23f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.312595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb768-ff01-4727-9567-41e209f23f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.314882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb768-ff01-4727-9567-41e209f23f73 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.330252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f124a0b2c2a4592ab5c907190b27508', 'request_id': 'req-c90550b9-afbf-4e09-a727-b154c555e901'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.437117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f124a0b2c2a4592ab5c907190b27508', 'request_id': 'req-c90550b9-afbf-4e09-a727-b154c555e901'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.442394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 955/1938] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:34.446151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb768-ff01-4727-9567-41e209f23f73 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.458622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f124a0b2c2a4592ab5c907190b27508', 'request_id': 'req-c90550b9-afbf-4e09-a727-b154c555e901'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.490839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] The token cache is being invalidate because project 2f124a0b2c2a4592ab5c907190b27508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:34.492864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.493638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c90550b9-afbf-4e09-a727-b154c555e901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:34.501495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 984/1939] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:34 2022] DELETE /identity/v3/projects/2f124a0b2c2a4592ab5c907190b27508 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:34.516790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4ed4-c50e-4762-9f9b-4c1899aa5b31 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.517214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4ed4-c50e-4762-9f9b-4c1899aa5b31 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.517378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4ed4-c50e-4762-9f9b-4c1899aa5b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.548486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-99fc4ed4-c50e-4762-9f9b-4c1899aa5b31 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:34.550076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-99fc4ed4-c50e-4762-9f9b-4c1899aa5b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:34.558387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb768-ff01-4727-9567-41e209f23f73 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.568571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 985/1940] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:34.571084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.571878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.572246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.572691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.573434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.574001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9a2bc-fa4e-4c15-bf76-0f54b183ed4c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.574385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9a2bc-fa4e-4c15-bf76-0f54b183ed4c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.574766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9a2bc-fa4e-4c15-bf76-0f54b183ed4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.575822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.606832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-96c9a2bc-fa4e-4c15-bf76-0f54b183ed4c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:34.608062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-96c9a2bc-fa4e-4c15-bf76-0f54b183ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:34.617186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 986/1941] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:34.621361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9a2bc-fa4e-4c15-bf76-0f54b183ed4c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.631589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9a2bc-fa4e-4c15-bf76-0f54b183ed4c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.641561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.641561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.641561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.641561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.642528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:34.642528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.642528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.656511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 987/1942] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:35:34.657520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.668624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9b3e1-c786-4621-a9d1-d8a7d769ea06 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.672904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3acaca-c9f5-4380-96a8-aed62c839224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.673562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3acaca-c9f5-4380-96a8-aed62c839224 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.673852 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3acaca-c9f5-4380-96a8-aed62c839224 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.674135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3acaca-c9f5-4380-96a8-aed62c839224 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.674883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3acaca-c9f5-4380-96a8-aed62c839224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.676778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3acaca-c9f5-4380-96a8-aed62c839224 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.695489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 956/1943] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:34.699415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.701077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:34.713665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c7df2-66aa-4dda-835f-1acc682c35f2 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.718467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168146a0-7a1f-4ffc-8f78-824df56d5baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.719121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168146a0-7a1f-4ffc-8f78-824df56d5baa admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.719406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168146a0-7a1f-4ffc-8f78-824df56d5baa admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.719687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168146a0-7a1f-4ffc-8f78-824df56d5baa admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.720448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168146a0-7a1f-4ffc-8f78-824df56d5baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.722195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168146a0-7a1f-4ffc-8f78-824df56d5baa admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.731917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 957/1944] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:34.735444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168146a0-7a1f-4ffc-8f78-824df56d5baa admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.744928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168146a0-7a1f-4ffc-8f78-824df56d5baa admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.749541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98e581-8233-4435-931b-f7393b7c7edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.750135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98e581-8233-4435-931b-f7393b7c7edc admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.750419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98e581-8233-4435-931b-f7393b7c7edc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.750695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98e581-8233-4435-931b-f7393b7c7edc admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.751383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98e581-8233-4435-931b-f7393b7c7edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.753439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98e581-8233-4435-931b-f7393b7c7edc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.763533 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 958/1945] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:34.767270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98e581-8233-4435-931b-f7393b7c7edc admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.771983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:34.775951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 988/1946] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/projects => generated 361 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:34.780207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3acaca-c9f5-4380-96a8-aed62c839224 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.780325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98e581-8233-4435-931b-f7393b7c7edc admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.837164 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a657e0-8146-4552-852a-4e26fb7d6776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.838124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a657e0-8146-4552-852a-4e26fb7d6776 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.838413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a657e0-8146-4552-852a-4e26fb7d6776 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.838692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a657e0-8146-4552-852a-4e26fb7d6776 admin admin] PATH_INFO: `/v3/projects/c143dfe0be6846548dbdd4040a1f2515/users/a389c2af5f214afebbf87748a31e1dae/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.841596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3acaca-c9f5-4380-96a8-aed62c839224 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.846119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e0523-c793-48ef-a47b-54bb781b27df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.847160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e0523-c793-48ef-a47b-54bb781b27df admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.847456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e0523-c793-48ef-a47b-54bb781b27df admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.847743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e0523-c793-48ef-a47b-54bb781b27df admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.848432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e0523-c793-48ef-a47b-54bb781b27df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.850442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e0523-c793-48ef-a47b-54bb781b27df admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.853175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a657e0-8146-4552-852a-4e26fb7d6776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c143dfe0be6846548dbdd4040a1f2515, user_id=a389c2af5f214afebbf87748a31e1dae, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.855803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a657e0-8146-4552-852a-4e26fb7d6776 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.930894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 959/1947] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:34 2022] PUT /identity/v3/projects/c143dfe0be6846548dbdd4040a1f2515/users/a389c2af5f214afebbf87748a31e1dae/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:34.943558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cafcb-fb0d-471b-85ba-b324a22323f2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.943851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cafcb-fb0d-471b-85ba-b324a22323f2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.944133 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cafcb-fb0d-471b-85ba-b324a22323f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.965914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 989/1948] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/users => generated 281 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:34.973084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e0523-c793-48ef-a47b-54bb781b27df admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:34.975706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-451cafcb-fb0d-471b-85ba-b324a22323f2 None None] MFA Rules not processed for user `a389c2af5f214afebbf87748a31e1dae`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:34.977126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-451cafcb-fb0d-471b-85ba-b324a22323f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:34.983442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e0523-c793-48ef-a47b-54bb781b27df admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:35.035871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:35.041007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 960/1949] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:35.042580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e730a-6195-4304-bdf3-f5c26563072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:35.043262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e730a-6195-4304-bdf3-f5c26563072b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:35.043557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e730a-6195-4304-bdf3-f5c26563072b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:35.044056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e730a-6195-4304-bdf3-f5c26563072b admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:35.044844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e730a-6195-4304-bdf3-f5c26563072b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:35.046573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e730a-6195-4304-bdf3-f5c26563072b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:35.053393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cafcb-fb0d-471b-85ba-b324a22323f2 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:35.054750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-451cafcb-fb0d-471b-85ba-b324a22323f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:35.057765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 990/1950] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:35.061946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e730a-6195-4304-bdf3-f5c26563072b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:35.072319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e730a-6195-4304-bdf3-f5c26563072b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:35.077032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc3df2-3896-473f-be62-7016eea52f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:35.077683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc3df2-3896-473f-be62-7016eea52f01 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:35.077971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc3df2-3896-473f-be62-7016eea52f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:35.078255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc3df2-3896-473f-be62-7016eea52f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:35.079048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc3df2-3896-473f-be62-7016eea52f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:35.080627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc3df2-3896-473f-be62-7016eea52f01 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:35.090455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 991/1951] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:35.094616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc3df2-3896-473f-be62-7016eea52f01 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:35.104691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc3df2-3896-473f-be62-7016eea52f01 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:35.109387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f622f9-bc0e-43f2-b5d8-7cc64e77f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:35.110265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f622f9-bc0e-43f2-b5d8-7cc64e77f219 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:35.110553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f622f9-bc0e-43f2-b5d8-7cc64e77f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:35.110849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f622f9-bc0e-43f2-b5d8-7cc64e77f219 admin admin] PATH_INFO: `/v3/projects/8af1525b3d3146e0bb286439712223a2/users/79e745c9b246471b82b29955e594a874/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:35.111987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cafcb-fb0d-471b-85ba-b324a22323f2 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:35.116392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562d7f2-7b4b-4282-bf35-21f84982e8a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:35.117115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562d7f2-7b4b-4282-bf35-21f84982e8a9 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:35.117405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562d7f2-7b4b-4282-bf35-21f84982e8a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:35.117686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562d7f2-7b4b-4282-bf35-21f84982e8a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:35.126359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f622f9-bc0e-43f2-b5d8-7cc64e77f219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af1525b3d3146e0bb286439712223a2, user_id=79e745c9b246471b82b29955e594a874, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:35.126519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562d7f2-7b4b-4282-bf35-21f84982e8a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:35.128433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562d7f2-7b4b-4282-bf35-21f84982e8a9 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:35.129766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f622f9-bc0e-43f2-b5d8-7cc64e77f219 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:35.137464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 961/1952] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:35.189518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 992/1953] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:35 2022] PUT /identity/v3/projects/8af1525b3d3146e0bb286439712223a2/users/79e745c9b246471b82b29955e594a874/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:35.200776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a38664-4840-416d-9068-dc417a57bceb None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:35.201072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a38664-4840-416d-9068-dc417a57bceb None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:35.201357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a38664-4840-416d-9068-dc417a57bceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:35.228127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-08a38664-4840-416d-9068-dc417a57bceb None None] MFA Rules not processed for user `79e745c9b246471b82b29955e594a874`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:35.229496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-08a38664-4840-416d-9068-dc417a57bceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:35.294965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 962/1954] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:36.053087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f622f9-bc0e-43f2-b5d8-7cc64e77f219 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:36.107980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f622f9-bc0e-43f2-b5d8-7cc64e77f219 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:36.111744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e1f3c-f968-427b-a02f-564c6de81558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:36.112473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e1f3c-f968-427b-a02f-564c6de81558 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:36.112818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e1f3c-f968-427b-a02f-564c6de81558 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:36.113106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e1f3c-f968-427b-a02f-564c6de81558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:36.121873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e1f3c-f968-427b-a02f-564c6de81558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:36.123456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e1f3c-f968-427b-a02f-564c6de81558 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:36.133331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 993/1955] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:36.147840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a38664-4840-416d-9068-dc417a57bceb None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:36.212947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a38664-4840-416d-9068-dc417a57bceb None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:36.217543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:36.218418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:36.218705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:36.218988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] PATH_INFO: `/v3/users/79e745c9b246471b82b29955e594a874` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:36.220219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e745c9b246471b82b29955e594a874)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:36.222232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:36.476963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:36.490738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e745c9b246471b82b29955e594a874'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:36.497858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e745c9b246471b82b29955e594a874'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:36.498609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e745c9b246471b82b29955e594a874'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:36.499284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e745c9b246471b82b29955e594a874'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:36.585197 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e745c9b246471b82b29955e594a874'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:36.606433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 963/1956] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:36 2022] DELETE /identity/v3/users/79e745c9b246471b82b29955e594a874 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:36.610993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7881c2-aef7-4751-8e43-62dfffdf586e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:36.611315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7881c2-aef7-4751-8e43-62dfffdf586e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:36.611599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7881c2-aef7-4751-8e43-62dfffdf586e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:36.629944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-ca7881c2-aef7-4751-8e43-62dfffdf586e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:36.631138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ca7881c2-aef7-4751-8e43-62dfffdf586e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:36.671753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:36.691261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 994/1957] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:36.701466 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:36.703289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:36.761939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f95a7-e3ec-435b-8b7d-fc90ec9cfde2 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:36.766054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:36.766764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:36.767053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:36.767337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:36.778503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:36.778636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:36.788795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 964/1958] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:37.235516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7881c2-aef7-4751-8e43-62dfffdf586e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:37.238784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ca7881c2-aef7-4751-8e43-62dfffdf586e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:37.250567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7881c2-aef7-4751-8e43-62dfffdf586e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:37.256076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:37.256995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:37.257285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:37.261561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] PATH_INFO: `/v3/projects/8af1525b3d3146e0bb286439712223a2` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:37.261561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af1525b3d3146e0bb286439712223a2)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:37.261561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:37.378287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af1525b3d3146e0bb286439712223a2', 'request_id': 'req-6eb3f741-d3b3-427d-8404-7586b4025428'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.473772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af1525b3d3146e0bb286439712223a2', 'request_id': 'req-6eb3f741-d3b3-427d-8404-7586b4025428'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.480258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af1525b3d3146e0bb286439712223a2', 'request_id': 'req-6eb3f741-d3b3-427d-8404-7586b4025428'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.494081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] The token cache is being invalidate because project 8af1525b3d3146e0bb286439712223a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:37.496646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.496646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6eb3f741-d3b3-427d-8404-7586b4025428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.499404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:37.499880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 995/1959] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:37 2022] DELETE /identity/v3/projects/8af1525b3d3146e0bb286439712223a2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:37.501031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:37.501300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df6749-3305-4653-a42f-9cbd45b2c69a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:37.501599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df6749-3305-4653-a42f-9cbd45b2c69a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:37.501888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df6749-3305-4653-a42f-9cbd45b2c69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:37.514175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3acf23-f4ab-4ee9-b3f5-47549d990de6 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:37.520630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-66df6749-3305-4653-a42f-9cbd45b2c69a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:37.522875 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-66df6749-3305-4653-a42f-9cbd45b2c69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:37.574183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:37.575073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:37.575486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:37.575774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] PATH_INFO: `/v3/users/4473832ed9654d40b4b2d30593e12c13` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:37.577138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4473832ed9654d40b4b2d30593e12c13)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:37.579066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:37.589289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 996/1960] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:37.598136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d9582-a987-4b8e-8f7b-89b9dbbadc92 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:37.598699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d9582-a987-4b8e-8f7b-89b9dbbadc92 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:37.598983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d9582-a987-4b8e-8f7b-89b9dbbadc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:37.604550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:37.617575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-aa0d9582-a987-4b8e-8f7b-89b9dbbadc92 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:37.618768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-aa0d9582-a987-4b8e-8f7b-89b9dbbadc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:37.627053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 997/1961] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:37.631651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d9582-a987-4b8e-8f7b-89b9dbbadc92 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:37.640404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d9582-a987-4b8e-8f7b-89b9dbbadc92 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:37.645640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:37.646335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:37.646622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:37.646897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:37.647619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:37.647925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:37.649530 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:37.661621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 998/1962] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:35:37.665901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:37.676047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c445753-c20b-4a15-b65d-f9b31801139c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:37.680968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:37.681625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:37.682585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:37.682585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:37.682863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:37.684990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:37.692326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:37.756638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4473832ed9654d40b4b2d30593e12c13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.763623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4473832ed9654d40b4b2d30593e12c13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.764280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4473832ed9654d40b4b2d30593e12c13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.764905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4473832ed9654d40b4b2d30593e12c13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.777956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 999/1963] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:37 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:37.782067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:37.783623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:37.825386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-90494e2e-2140-4393-8a7d-861d055fa679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4473832ed9654d40b4b2d30593e12c13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:37.844621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 965/1964] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:37 2022] DELETE /identity/v3/users/4473832ed9654d40b4b2d30593e12c13 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:37.848735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca5bfe-b538-46c4-b26c-351674ed45ce admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:37.848902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439b66b-7300-4c77-b0ee-5c48641e1ac6 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:37.849209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439b66b-7300-4c77-b0ee-5c48641e1ac6 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:37.849625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439b66b-7300-4c77-b0ee-5c48641e1ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:37.867476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e439b66b-7300-4c77-b0ee-5c48641e1ac6 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:37.868730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e439b66b-7300-4c77-b0ee-5c48641e1ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:37.904766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c8b5f-e7f8-4389-91bb-f745493a613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:37.905444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8b5f-e7f8-4389-91bb-f745493a613e admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:37.905729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8b5f-e7f8-4389-91bb-f745493a613e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:37.906007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8b5f-e7f8-4389-91bb-f745493a613e admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:37.906733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c8b5f-e7f8-4389-91bb-f745493a613e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:37.908613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c8b5f-e7f8-4389-91bb-f745493a613e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:37.927271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 966/1965] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:37.937379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439b66b-7300-4c77-b0ee-5c48641e1ac6 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:37.938727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e439b66b-7300-4c77-b0ee-5c48641e1ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:38.000158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439b66b-7300-4c77-b0ee-5c48641e1ac6 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.004720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c38099-329a-46a6-a727-1fa5d09ee0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.005528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c38099-329a-46a6-a727-1fa5d09ee0ba service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.005916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c38099-329a-46a6-a727-1fa5d09ee0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.006424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c38099-329a-46a6-a727-1fa5d09ee0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.020806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c38099-329a-46a6-a727-1fa5d09ee0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.022254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c38099-329a-46a6-a727-1fa5d09ee0ba service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.028495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1000/1966] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:37 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:38.032923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c8b5f-e7f8-4389-91bb-f745493a613e admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.036145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 967/1967] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:38.047738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c8b5f-e7f8-4389-91bb-f745493a613e admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.054701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dab591-96ea-4ebe-96f5-2eda26aea173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.055488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dab591-96ea-4ebe-96f5-2eda26aea173 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.055954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dab591-96ea-4ebe-96f5-2eda26aea173 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.056265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dab591-96ea-4ebe-96f5-2eda26aea173 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.057991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dab591-96ea-4ebe-96f5-2eda26aea173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.059152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dab591-96ea-4ebe-96f5-2eda26aea173 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.075666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1001/1968] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:38.076979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c38099-329a-46a6-a727-1fa5d09ee0ba service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.097066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c38099-329a-46a6-a727-1fa5d09ee0ba service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.102049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbfff8-0fa8-49bf-87c8-b0f72a1def0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.102739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbfff8-0fa8-49bf-87c8-b0f72a1def0b admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.103049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbfff8-0fa8-49bf-87c8-b0f72a1def0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.103335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbfff8-0fa8-49bf-87c8-b0f72a1def0b admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.104301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbfff8-0fa8-49bf-87c8-b0f72a1def0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.106177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbfff8-0fa8-49bf-87c8-b0f72a1def0b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.123130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dab591-96ea-4ebe-96f5-2eda26aea173 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.123592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 968/1969] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:38.136300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dab591-96ea-4ebe-96f5-2eda26aea173 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.145543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35fa22-e9ae-4749-9c4d-21a9bca36d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.146517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35fa22-e9ae-4749-9c4d-21a9bca36d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.146808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35fa22-e9ae-4749-9c4d-21a9bca36d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.147094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35fa22-e9ae-4749-9c4d-21a9bca36d43 admin admin] PATH_INFO: `/v3/projects/9e5f416610d34598b7ca363bf829a1f0/users/6ef59f722e0c46b19eb4aca28f4b0a02/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.164130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35fa22-e9ae-4749-9c4d-21a9bca36d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5f416610d34598b7ca363bf829a1f0, user_id=6ef59f722e0c46b19eb4aca28f4b0a02, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.167409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35fa22-e9ae-4749-9c4d-21a9bca36d43 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.280899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1002/1970] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:38 2022] PUT /identity/v3/projects/9e5f416610d34598b7ca363bf829a1f0/users/6ef59f722e0c46b19eb4aca28f4b0a02/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:38.292353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410c22b-9e7f-4061-8801-bb5821f63140 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.292682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410c22b-9e7f-4061-8801-bb5821f63140 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.293419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410c22b-9e7f-4061-8801-bb5821f63140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.324341 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-0410c22b-9e7f-4061-8801-bb5821f63140 None None] MFA Rules not processed for user `6ef59f722e0c46b19eb4aca28f4b0a02`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:38.325590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-0410c22b-9e7f-4061-8801-bb5821f63140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:38.404221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 969/1971] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:38.408922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35fa22-e9ae-4749-9c4d-21a9bca36d43 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.485601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35fa22-e9ae-4749-9c4d-21a9bca36d43 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.490469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.491139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.491443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.491725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.493722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.496213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.533828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:38.618636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1003/1972] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:38 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:38.619746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410c22b-9e7f-4061-8801-bb5821f63140 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.631013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410c22b-9e7f-4061-8801-bb5821f63140 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.687982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167f16c-6cb6-4777-a908-2fc9ce68011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.688723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167f16c-6cb6-4777-a908-2fc9ce68011e admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.689274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167f16c-6cb6-4777-a908-2fc9ce68011e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.689554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167f16c-6cb6-4777-a908-2fc9ce68011e admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.690874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167f16c-6cb6-4777-a908-2fc9ce68011e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.692911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167f16c-6cb6-4777-a908-2fc9ce68011e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.761680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.763399 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:38.776382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e414c-f374-4e7c-9dd3-3b3c2636e4e9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.781272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.782141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.782466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.782881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] PATH_INFO: `/v3/projects/cdf2e66adadf458da01c96754653686e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.784091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf2e66adadf458da01c96754653686e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.786107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.807032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 970/1973] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:38 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:38.810151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167f16c-6cb6-4777-a908-2fc9ce68011e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.824636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167f16c-6cb6-4777-a908-2fc9ce68011e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.830383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea9946-b2a3-4c13-a26f-317270c45dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.831130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea9946-b2a3-4c13-a26f-317270c45dfa admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.831490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea9946-b2a3-4c13-a26f-317270c45dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.831846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea9946-b2a3-4c13-a26f-317270c45dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.832734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea9946-b2a3-4c13-a26f-317270c45dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.834436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea9946-b2a3-4c13-a26f-317270c45dfa admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.844683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 971/1974] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:38.848649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea9946-b2a3-4c13-a26f-317270c45dfa admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.858857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea9946-b2a3-4c13-a26f-317270c45dfa admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.864189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ffb44-b7d3-4467-b261-ae40e46eec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.864884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ffb44-b7d3-4467-b261-ae40e46eec74 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.865186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ffb44-b7d3-4467-b261-ae40e46eec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.865447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ffb44-b7d3-4467-b261-ae40e46eec74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.866177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ffb44-b7d3-4467-b261-ae40e46eec74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.867796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ffb44-b7d3-4467-b261-ae40e46eec74 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.879201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 972/1975] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:38.884041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ffb44-b7d3-4467-b261-ae40e46eec74 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:38.893159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ffb44-b7d3-4467-b261-ae40e46eec74 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:38.897987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f58913-9615-407b-987b-7356b7637def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:38.898964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f58913-9615-407b-987b-7356b7637def admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:38.899251 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f58913-9615-407b-987b-7356b7637def admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:38.899527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f58913-9615-407b-987b-7356b7637def admin admin] PATH_INFO: `/v3/projects/deddebabc1f441cdaf2d6ba49ffd461c/users/6570c1936d5d4a68b7ab476da2153ddc/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:38.916865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f58913-9615-407b-987b-7356b7637def admin admin] RBAC: Authorizing `identity:create_grant(project_id=deddebabc1f441cdaf2d6ba49ffd461c, user_id=6570c1936d5d4a68b7ab476da2153ddc, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:38.919961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f58913-9615-407b-987b-7356b7637def admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:38.942535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf2e66adadf458da01c96754653686e', 'request_id': 'req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.057359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf2e66adadf458da01c96754653686e', 'request_id': 'req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.068841 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf2e66adadf458da01c96754653686e', 'request_id': 'req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.071604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 973/1976] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:38 2022] PUT /identity/v3/projects/deddebabc1f441cdaf2d6ba49ffd461c/users/6570c1936d5d4a68b7ab476da2153ddc/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:39.085687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c61cf-c6cb-4d68-a24d-b35402ba3115 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:39.085887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] The token cache is being invalidate because project cdf2e66adadf458da01c96754653686e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:39.086178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.087095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.087520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c61cf-c6cb-4d68-a24d-b35402ba3115 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:39.087896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c61cf-c6cb-4d68-a24d-b35402ba3115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:39.088982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1004/1977] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:38 2022] DELETE /identity/v3/projects/cdf2e66adadf458da01c96754653686e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:39.106937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:39.108581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:39.126744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-602c61cf-c6cb-4d68-a24d-b35402ba3115 None None] MFA Rules not processed for user `6570c1936d5d4a68b7ab476da2153ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:39.128652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-602c61cf-c6cb-4d68-a24d-b35402ba3115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:39.206758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 974/1978] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:39.211326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c61cf-c6cb-4d68-a24d-b35402ba3115 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:39.212988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-602c61cf-c6cb-4d68-a24d-b35402ba3115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:39.218717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32534cd-f7ee-4f68-b7b1-b6a0a4a6b1c8 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:39.286444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c61cf-c6cb-4d68-a24d-b35402ba3115 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:39.291963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da81e35-51ac-4f14-8538-140af58fb123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:39.292693 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81e35-51ac-4f14-8538-140af58fb123 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:39.292982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81e35-51ac-4f14-8538-140af58fb123 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:39.293260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81e35-51ac-4f14-8538-140af58fb123 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:39.293963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da81e35-51ac-4f14-8538-140af58fb123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:39.296178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da81e35-51ac-4f14-8538-140af58fb123 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:39.298788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:39.299947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:39.300306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:39.300635 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] PATH_INFO: `/v3/users/6e8baac43ea54c36b45e2dc01d0f8f24` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:39.302024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8baac43ea54c36b45e2dc01d0f8f24)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:39.304017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:39.316624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:39.412331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 975/1979] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:39 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:39.423009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da81e35-51ac-4f14-8538-140af58fb123 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:39.514694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da81e35-51ac-4f14-8538-140af58fb123 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:39.519886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:39.520607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:39.520900 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:39.521183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:39.521889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:39.523845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:39.553972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8baac43ea54c36b45e2dc01d0f8f24'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.566886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8baac43ea54c36b45e2dc01d0f8f24'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.574240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8baac43ea54c36b45e2dc01d0f8f24'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.574586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8baac43ea54c36b45e2dc01d0f8f24'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.575269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8baac43ea54c36b45e2dc01d0f8f24'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:39.620994 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 976/1980] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:39 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:39.624797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:39.626432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:39.640434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3ee31-6738-4717-984d-5ed93a15705a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:39.645561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1005/1981] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:39 2022] DELETE /identity/v3/users/6e8baac43ea54c36b45e2dc01d0f8f24 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:39.650021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c207bb-e150-4bbc-acc1-a773b09ab65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:39.650741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c207bb-e150-4bbc-acc1-a773b09ab65d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:39.651033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c207bb-e150-4bbc-acc1-a773b09ab65d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:39.651321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c207bb-e150-4bbc-acc1-a773b09ab65d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:39.652068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c207bb-e150-4bbc-acc1-a773b09ab65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:39.653750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c207bb-e150-4bbc-acc1-a773b09ab65d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:39.664657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 977/1982] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:39.669923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:39.747640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26c82-4450-43ce-8941-4def3107bf9c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:39.756380 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc42e28-714f-4694-8b9f-4b7998b1b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:39.757644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc42e28-714f-4694-8b9f-4b7998b1b30c admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:39.758542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc42e28-714f-4694-8b9f-4b7998b1b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:39.760247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc42e28-714f-4694-8b9f-4b7998b1b30c admin admin] PATH_INFO: `/v3/projects/561cec281d334770b041f277fd994841/users/125b1e83c22c4ecab47428ee5b927fc4/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:39.782550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc42e28-714f-4694-8b9f-4b7998b1b30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=561cec281d334770b041f277fd994841, user_id=125b1e83c22c4ecab47428ee5b927fc4, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:39.785312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc42e28-714f-4694-8b9f-4b7998b1b30c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:39.866213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1006/1983] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:39 2022] PUT /identity/v3/projects/561cec281d334770b041f277fd994841/users/125b1e83c22c4ecab47428ee5b927fc4/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:39.871149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c207bb-e150-4bbc-acc1-a773b09ab65d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:39.895510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c207bb-e150-4bbc-acc1-a773b09ab65d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:39.960060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15644d4-5e44-470e-9732-8b47e9a7ec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:39.960921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15644d4-5e44-470e-9732-8b47e9a7ec1d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:39.961204 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15644d4-5e44-470e-9732-8b47e9a7ec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:39.961598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15644d4-5e44-470e-9732-8b47e9a7ec1d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:39.962423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15644d4-5e44-470e-9732-8b47e9a7ec1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:39.964032 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15644d4-5e44-470e-9732-8b47e9a7ec1d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:39.976070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 978/1984] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:39.980557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc42e28-714f-4694-8b9f-4b7998b1b30c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:39.992333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc42e28-714f-4694-8b9f-4b7998b1b30c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:39.996911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:39.997814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:39.998105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:39.998389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] PATH_INFO: `/v3/projects/561cec281d334770b041f277fd994841/users/125b1e83c22c4ecab47428ee5b927fc4/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.001135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561cec281d334770b041f277fd994841, user_id=125b1e83c22c4ecab47428ee5b927fc4, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:40.003974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:40.015538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1007/1985] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:39 2022] PUT /identity/v3/projects/561cec281d334770b041f277fd994841/users/125b1e83c22c4ecab47428ee5b927fc4/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:40.032258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea2bda-20a3-4cb7-a4f1-06d585ff2c53 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.032598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea2bda-20a3-4cb7-a4f1-06d585ff2c53 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.032887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea2bda-20a3-4cb7-a4f1-06d585ff2c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.066888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9dea2bda-20a3-4cb7-a4f1-06d585ff2c53 None None] MFA Rules not processed for user `125b1e83c22c4ecab47428ee5b927fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:40.068443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9dea2bda-20a3-4cb7-a4f1-06d585ff2c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:40.114955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:40.116808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:40.138215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:40.140554 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 979/1986] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:40.154789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea2bda-20a3-4cb7-a4f1-06d585ff2c53 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:40.157524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9dea2bda-20a3-4cb7-a4f1-06d585ff2c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:40.192027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:40.207138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c252f9-139d-40df-bf3d-0b6ff5335fa8 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:40.212470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:40.214815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.214815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.214815 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] PATH_INFO: `/v3/projects/8b9e046016b246c18add6b4ddc28ece1` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.215284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9e046016b246c18add6b4ddc28ece1)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:40.217258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:40.222971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea2bda-20a3-4cb7-a4f1-06d585ff2c53 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:40.227646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ccd55-4dfa-42ac-ab0b-8da204349b31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:40.227775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ccd55-4dfa-42ac-ab0b-8da204349b31 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.228028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ccd55-4dfa-42ac-ab0b-8da204349b31 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.228302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ccd55-4dfa-42ac-ab0b-8da204349b31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.230379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-741ccd55-4dfa-42ac-ab0b-8da204349b31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:40.291476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ccd55-4dfa-42ac-ab0b-8da204349b31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:40.293015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ccd55-4dfa-42ac-ab0b-8da204349b31 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:40.302453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 980/1987] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:40.367642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9e046016b246c18add6b4ddc28ece1', 'request_id': 'req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:40.451026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9e046016b246c18add6b4ddc28ece1', 'request_id': 'req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:40.459517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9e046016b246c18add6b4ddc28ece1', 'request_id': 'req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:40.472350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] The token cache is being invalidate because project 8b9e046016b246c18add6b4ddc28ece1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:40.472918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:40.473777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18976a1e-06e2-4a80-84c0-0e1f23f06fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:40.475561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1008/1988] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:40 2022] DELETE /identity/v3/projects/8b9e046016b246c18add6b4ddc28ece1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:40.480633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44871c13-c6f5-478c-a60d-6b6eea5ef94c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.481232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44871c13-c6f5-478c-a60d-6b6eea5ef94c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.481592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44871c13-c6f5-478c-a60d-6b6eea5ef94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.503929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-44871c13-c6f5-478c-a60d-6b6eea5ef94c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:40.505288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-44871c13-c6f5-478c-a60d-6b6eea5ef94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:40.576574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 981/1989] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:40.591045 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3acc84-8e10-43ae-beb4-b3fcbf72d1aa None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.591431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3acc84-8e10-43ae-beb4-b3fcbf72d1aa None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.591802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3acc84-8e10-43ae-beb4-b3fcbf72d1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.611622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-fd3acc84-8e10-43ae-beb4-b3fcbf72d1aa None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:40.613061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-fd3acc84-8e10-43ae-beb4-b3fcbf72d1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:40.621827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1009/1990] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:40.626189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44871c13-c6f5-478c-a60d-6b6eea5ef94c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:40.639827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44871c13-c6f5-478c-a60d-6b6eea5ef94c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:40.646716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:40.648320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.648320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.648320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.648922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:40.649478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:40.651238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:40.674381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 982/1991] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:40.675720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3acc84-8e10-43ae-beb4-b3fcbf72d1aa None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:40.686650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3acc84-8e10-43ae-beb4-b3fcbf72d1aa None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:40.692861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c6791-df96-4389-9431-5384bec19a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:40.693657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c6791-df96-4389-9431-5384bec19a8f admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.694016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c6791-df96-4389-9431-5384bec19a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.694388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c6791-df96-4389-9431-5384bec19a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.695184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c6791-df96-4389-9431-5384bec19a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:40.697251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c6791-df96-4389-9431-5384bec19a8f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:40.790191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1010/1992] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:40 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:40.791459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:40.801416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955de5d-a043-4724-b556-dd43496b04c4 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:40.859726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9d28d-54c2-4fb7-bbe2-b7af6133851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:40.860405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9d28d-54c2-4fb7-bbe2-b7af6133851f admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.860723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9d28d-54c2-4fb7-bbe2-b7af6133851f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.861395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9d28d-54c2-4fb7-bbe2-b7af6133851f admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.862112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9d28d-54c2-4fb7-bbe2-b7af6133851f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:40.864325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9d28d-54c2-4fb7-bbe2-b7af6133851f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:40.965116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 983/1993] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:40 2022] POST /identity/v3/users => generated 282 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:40.966486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c6791-df96-4389-9431-5384bec19a8f admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:40.978635 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c6791-df96-4389-9431-5384bec19a8f admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:40.992604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6efd1-04b7-4496-a477-67aaab4e242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:40.993309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6efd1-04b7-4496-a477-67aaab4e242e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:40.993599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6efd1-04b7-4496-a477-67aaab4e242e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:40.993881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6efd1-04b7-4496-a477-67aaab4e242e admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:40.994708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6efd1-04b7-4496-a477-67aaab4e242e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:40.996365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6efd1-04b7-4496-a477-67aaab4e242e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.007825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1011/1994] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:41.011955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9d28d-54c2-4fb7-bbe2-b7af6133851f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:41.022194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9d28d-54c2-4fb7-bbe2-b7af6133851f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:41.027001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b2c36-7133-47b9-9156-cade90646551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:41.027669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b2c36-7133-47b9-9156-cade90646551 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.027957 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b2c36-7133-47b9-9156-cade90646551 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.028240 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b2c36-7133-47b9-9156-cade90646551 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.029348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b2c36-7133-47b9-9156-cade90646551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:41.030843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b2c36-7133-47b9-9156-cade90646551 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.048346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 984/1995] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:41.053348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6efd1-04b7-4496-a477-67aaab4e242e admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:41.063780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6efd1-04b7-4496-a477-67aaab4e242e admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:41.070010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db81604-690e-4a62-974a-2dadd15941c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:41.070969 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db81604-690e-4a62-974a-2dadd15941c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.071295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db81604-690e-4a62-974a-2dadd15941c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.071638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db81604-690e-4a62-974a-2dadd15941c0 admin admin] PATH_INFO: `/v3/projects/01ad832b338d4b5c83c57fcec7b37d58/users/326ca8df7a744543bb4f444636fb400b/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.088293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db81604-690e-4a62-974a-2dadd15941c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ad832b338d4b5c83c57fcec7b37d58, user_id=326ca8df7a744543bb4f444636fb400b, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:41.091572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db81604-690e-4a62-974a-2dadd15941c0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.184147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1012/1996] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:41 2022] PUT /identity/v3/projects/01ad832b338d4b5c83c57fcec7b37d58/users/326ca8df7a744543bb4f444636fb400b/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:41.195718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e7e5-b471-4d4a-bcd8-9848c0214bab None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.196006 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e7e5-b471-4d4a-bcd8-9848c0214bab None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.196290 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e7e5-b471-4d4a-bcd8-9848c0214bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.206635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:41.226852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-1879e7e5-b471-4d4a-bcd8-9848c0214bab None None] MFA Rules not processed for user `326ca8df7a744543bb4f444636fb400b`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:41.228058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1879e7e5-b471-4d4a-bcd8-9848c0214bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:41.274514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:41.296188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 985/1997] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:41 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:41.301338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db81604-690e-4a62-974a-2dadd15941c0 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:41.369068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db81604-690e-4a62-974a-2dadd15941c0 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:41.373896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd141f6-c54f-4e91-91af-153725b8adc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:41.374569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd141f6-c54f-4e91-91af-153725b8adc1 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.374860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd141f6-c54f-4e91-91af-153725b8adc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.375144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd141f6-c54f-4e91-91af-153725b8adc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.375839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd141f6-c54f-4e91-91af-153725b8adc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:41.380733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd141f6-c54f-4e91-91af-153725b8adc1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.491946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1013/1998] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:41 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:41.496055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879e7e5-b471-4d4a-bcd8-9848c0214bab None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:41.507212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879e7e5-b471-4d4a-bcd8-9848c0214bab None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:41.576799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e99852-8fec-4f38-8908-e452f89bc9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:41.577621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e99852-8fec-4f38-8908-e452f89bc9ef admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.577973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e99852-8fec-4f38-8908-e452f89bc9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.578320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e99852-8fec-4f38-8908-e452f89bc9ef admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.579237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e99852-8fec-4f38-8908-e452f89bc9ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:41.581334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e99852-8fec-4f38-8908-e452f89bc9ef admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.597054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:41.648400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 986/1999] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:41 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:41.652412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd141f6-c54f-4e91-91af-153725b8adc1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:41.664917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd141f6-c54f-4e91-91af-153725b8adc1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:41.672108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b4e1-45ac-4be2-9aa1-bba54f4eec65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:41.672932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b4e1-45ac-4be2-9aa1-bba54f4eec65 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.675561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b4e1-45ac-4be2-9aa1-bba54f4eec65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.675561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b4e1-45ac-4be2-9aa1-bba54f4eec65 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.675561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2b4e1-45ac-4be2-9aa1-bba54f4eec65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:41.676482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2b4e1-45ac-4be2-9aa1-bba54f4eec65 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.687563 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1014/2000] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:41.692187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e99852-8fec-4f38-8908-e452f89bc9ef admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:41.706385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e99852-8fec-4f38-8908-e452f89bc9ef admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:41.707523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39067abb-ca73-42a7-86f4-5ff320107681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:41.708410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39067abb-ca73-42a7-86f4-5ff320107681 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.708751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39067abb-ca73-42a7-86f4-5ff320107681 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.709035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39067abb-ca73-42a7-86f4-5ff320107681 admin admin] PATH_INFO: `/v3/projects/3f20a3fdd59b4fc2a36646bb828ef398/users/10de979bd76e4b0a8ec6d958abe81d93/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.724023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39067abb-ca73-42a7-86f4-5ff320107681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f20a3fdd59b4fc2a36646bb828ef398, user_id=10de979bd76e4b0a8ec6d958abe81d93, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:41.727252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39067abb-ca73-42a7-86f4-5ff320107681 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.785946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 987/2001] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:41 2022] PUT /identity/v3/projects/3f20a3fdd59b4fc2a36646bb828ef398/users/10de979bd76e4b0a8ec6d958abe81d93/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:41.789977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b4e1-45ac-4be2-9aa1-bba54f4eec65 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:41.791028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:41.800420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b4e1-45ac-4be2-9aa1-bba54f4eec65 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:41.833033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:41.859896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f76d6c-6083-484c-9ff5-3c83e078ce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:41.860612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f76d6c-6083-484c-9ff5-3c83e078ce2e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.860902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f76d6c-6083-484c-9ff5-3c83e078ce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.861184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f76d6c-6083-484c-9ff5-3c83e078ce2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.861918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f76d6c-6083-484c-9ff5-3c83e078ce2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:41.863588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f76d6c-6083-484c-9ff5-3c83e078ce2e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.874772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1015/2002] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:41.878726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39067abb-ca73-42a7-86f4-5ff320107681 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:41.889458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39067abb-ca73-42a7-86f4-5ff320107681 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:41.894645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:41.895597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.899923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.899923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] PATH_INFO: `/v3/projects/3f20a3fdd59b4fc2a36646bb828ef398/users/10de979bd76e4b0a8ec6d958abe81d93/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.899923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f20a3fdd59b4fc2a36646bb828ef398, user_id=10de979bd76e4b0a8ec6d958abe81d93, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:41.900909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:41.912250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 988/2003] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:41 2022] PUT /identity/v3/projects/3f20a3fdd59b4fc2a36646bb828ef398/users/10de979bd76e4b0a8ec6d958abe81d93/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:41.923351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d02fa8-d059-40d0-bb9c-53358c3fef60 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:41.923644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d02fa8-d059-40d0-bb9c-53358c3fef60 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:41.923956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d02fa8-d059-40d0-bb9c-53358c3fef60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:41.952318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-46d02fa8-d059-40d0-bb9c-53358c3fef60 None None] MFA Rules not processed for user `10de979bd76e4b0a8ec6d958abe81d93`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:41.953608 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-46d02fa8-d059-40d0-bb9c-53358c3fef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:42.043604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1016/2004] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:42.051067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:42.052816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:42.070819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:42.112930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf397b2-c58b-4529-b2f6-1e245f41dbab admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:42.116627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:42.116766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:42.117309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:42.117600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:42.117879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:42.126801 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:42.128754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:42.139184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 989/2005] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:42.347091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d02fa8-d059-40d0-bb9c-53358c3fef60 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:42.356441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d02fa8-d059-40d0-bb9c-53358c3fef60 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:42.361201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9370696-2eaa-427e-ba15-1d965823875d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:42.361844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9370696-2eaa-427e-ba15-1d965823875d service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:42.362132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9370696-2eaa-427e-ba15-1d965823875d service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:42.362415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9370696-2eaa-427e-ba15-1d965823875d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:42.422389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9370696-2eaa-427e-ba15-1d965823875d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:42.423894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9370696-2eaa-427e-ba15-1d965823875d service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:42.432907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1017/2006] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:42 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:42.459568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:42.506570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:42.665119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:42.989164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:43.111704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:43.472130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:43.714180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:43.817136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:43.887893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:43.967845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:44.114541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:44.280345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:44.399867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:44.399867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:44.399867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:44.410014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcd320-d909-4ee2-b6ba-6ee59fd8d054 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:44.413687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:44.414333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:44.414620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:44.414904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:44.416684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:44.478699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:44.480170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:44.490799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 990/2007] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:44.714382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:45.137039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:45.185335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:49.140415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9370696-2eaa-427e-ba15-1d965823875d service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:49.150555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9370696-2eaa-427e-ba15-1d965823875d service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:49.154366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afcf1a-9ab6-4f4a-8927-bfb147d6f527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:49.155012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afcf1a-9ab6-4f4a-8927-bfb147d6f527 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:49.155300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afcf1a-9ab6-4f4a-8927-bfb147d6f527 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:49.155579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afcf1a-9ab6-4f4a-8927-bfb147d6f527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:49.157425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-27afcf1a-9ab6-4f4a-8927-bfb147d6f527 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:49.206366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:49.232568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afcf1a-9ab6-4f4a-8927-bfb147d6f527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:49.234272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afcf1a-9ab6-4f4a-8927-bfb147d6f527 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:49.258687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1018/2008] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:52.339495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:52.414074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7bb9c-071b-44ef-85f2-b47bfdb9f0d9 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:52.420743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:52.420743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:52.420743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:52.420743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] PATH_INFO: `/v3/users/326ca8df7a744543bb4f444636fb400b` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:52.422071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=326ca8df7a744543bb4f444636fb400b)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:52.424008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:52.625121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326ca8df7a744543bb4f444636fb400b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:52.632925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326ca8df7a744543bb4f444636fb400b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:52.633604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326ca8df7a744543bb4f444636fb400b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:52.634213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326ca8df7a744543bb4f444636fb400b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:52.719702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326ca8df7a744543bb4f444636fb400b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:52.738529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 991/2009] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:52 2022] DELETE /identity/v3/users/326ca8df7a744543bb4f444636fb400b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:52.743178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70df6d-f565-449b-ab58-ab221d599c0b None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:52.743624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70df6d-f565-449b-ab58-ab221d599c0b None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:52.743738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70df6d-f565-449b-ab58-ab221d599c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:52.763215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1b70df6d-f565-449b-ab58-ab221d599c0b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:52.764590 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1b70df6d-f565-449b-ab58-ab221d599c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:52.837025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1019/2010] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:52.849682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:52.851595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:52.916205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:52.921331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb7810-00e0-45e4-9822-f2847be288ae admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:52.925475 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370f715-fdd2-4769-84a7-c14b0fe9126c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:52.926125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370f715-fdd2-4769-84a7-c14b0fe9126c service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:52.926409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370f715-fdd2-4769-84a7-c14b0fe9126c service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:52.926687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370f715-fdd2-4769-84a7-c14b0fe9126c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:52.935525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370f715-fdd2-4769-84a7-c14b0fe9126c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:52.936980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370f715-fdd2-4769-84a7-c14b0fe9126c service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:52.947369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 992/2011] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:53.397032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70df6d-f565-449b-ab58-ab221d599c0b None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:53.397032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1b70df6d-f565-449b-ab58-ab221d599c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:53.418928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70df6d-f565-449b-ab58-ab221d599c0b None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:53.425883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:53.425883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:53.426108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:53.426246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] PATH_INFO: `/v3/projects/01ad832b338d4b5c83c57fcec7b37d58` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:53.427926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ad832b338d4b5c83c57fcec7b37d58)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:53.430294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:53.552685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ad832b338d4b5c83c57fcec7b37d58', 'request_id': 'req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:53.619113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ad832b338d4b5c83c57fcec7b37d58', 'request_id': 'req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:53.625609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ad832b338d4b5c83c57fcec7b37d58', 'request_id': 'req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:53.637039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] The token cache is being invalidate because project 01ad832b338d4b5c83c57fcec7b37d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:53.637333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:53.637967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:53.639483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1020/2012] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:53 2022] DELETE /identity/v3/projects/01ad832b338d4b5c83c57fcec7b37d58 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:53.643544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370f715-fdd2-4769-84a7-c14b0fe9126c service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:53.654800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370f715-fdd2-4769-84a7-c14b0fe9126c service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:53.712633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:53.714472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:53.714472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:53.714789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] PATH_INFO: `/v3/users/10de979bd76e4b0a8ec6d958abe81d93` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:53.716775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] RBAC: Authorizing `identity:delete_user(user_id=10de979bd76e4b0a8ec6d958abe81d93)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:53.720427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:53.968544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10de979bd76e4b0a8ec6d958abe81d93'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:53.976908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10de979bd76e4b0a8ec6d958abe81d93'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:53.977626 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10de979bd76e4b0a8ec6d958abe81d93'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:53.978231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10de979bd76e4b0a8ec6d958abe81d93'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.082665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10de979bd76e4b0a8ec6d958abe81d93'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.103558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 993/2013] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:53 2022] DELETE /identity/v3/users/10de979bd76e4b0a8ec6d958abe81d93 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:54.171461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:54.173417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:54.187606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef301-7367-4a80-a7cc-4535e1fec8ad admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:54.251018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:54.251965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:54.252250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:54.253057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] PATH_INFO: `/v3/users/6ef59f722e0c46b19eb4aca28f4b0a02` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:54.254416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef59f722e0c46b19eb4aca28f4b0a02)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:54.256346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:54.258961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:54.462237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef59f722e0c46b19eb4aca28f4b0a02'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.477829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef59f722e0c46b19eb4aca28f4b0a02'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.490071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef59f722e0c46b19eb4aca28f4b0a02'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.490931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef59f722e0c46b19eb4aca28f4b0a02'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.491758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f1d94d2d-6326-4688-9522-3950be2ef9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef59f722e0c46b19eb4aca28f4b0a02'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.568092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:54.569750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:54.596453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74062d44-1ae9-46b7-80f0-9ebaa28dbdde admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:54.602326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:54.603371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:54.603753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:54.604219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] PATH_INFO: `/v3/projects/3f20a3fdd59b4fc2a36646bb828ef398` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:54.605602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f20a3fdd59b4fc2a36646bb828ef398)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:54.608493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:54.631902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1021/2014] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:54 2022] DELETE /identity/v3/users/6ef59f722e0c46b19eb4aca28f4b0a02 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:54.636317 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db704-3e95-4788-a7af-080bb7e9694c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:54.636650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db704-3e95-4788-a7af-080bb7e9694c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:54.636930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db704-3e95-4788-a7af-080bb7e9694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:54.662366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-003db704-3e95-4788-a7af-080bb7e9694c None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:54.663656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-003db704-3e95-4788-a7af-080bb7e9694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:54.726346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1022/2015] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:54.736093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003db704-3e95-4788-a7af-080bb7e9694c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:54.738090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-003db704-3e95-4788-a7af-080bb7e9694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:54.746434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f20a3fdd59b4fc2a36646bb828ef398', 'request_id': 'req-f9d3af8a-74db-48e8-af08-d23ece3d298f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.798129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003db704-3e95-4788-a7af-080bb7e9694c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:54.813280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f20a3fdd59b4fc2a36646bb828ef398', 'request_id': 'req-f9d3af8a-74db-48e8-af08-d23ece3d298f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.819492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:54.823706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f20a3fdd59b4fc2a36646bb828ef398', 'request_id': 'req-f9d3af8a-74db-48e8-af08-d23ece3d298f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.832485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] The token cache is being invalidate because project 3f20a3fdd59b4fc2a36646bb828ef398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:54.832809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.833480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f9d3af8a-74db-48e8-af08-d23ece3d298f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:54.835033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 994/2016] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:54 2022] DELETE /identity/v3/projects/3f20a3fdd59b4fc2a36646bb828ef398 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:54.839644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c4af8-c5d8-4596-b8b6-a0bf46984969 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:54.839933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c4af8-c5d8-4596-b8b6-a0bf46984969 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:54.840449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c4af8-c5d8-4596-b8b6-a0bf46984969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:54.851428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:54.852095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:54.852572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:54.852865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:54.854707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:54.858590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-511c4af8-c5d8-4596-b8b6-a0bf46984969 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:54.859763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-511c4af8-c5d8-4596-b8b6-a0bf46984969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:54.923117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:54.924474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 995/2017] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:54.929075 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:54.933880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3a6ed-4e67-4b64-a3e5-0b81c9cc9ff0 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:54.933880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3a6ed-4e67-4b64-a3e5-0b81c9cc9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:54.933880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3a6ed-4e67-4b64-a3e5-0b81c9cc9ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:54.943910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1023/2018] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:54.954386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-42a3a6ed-4e67-4b64-a3e5-0b81c9cc9ff0 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:54.955673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-42a3a6ed-4e67-4b64-a3e5-0b81c9cc9ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:54.964557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 996/2019] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:54.968495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:54.979120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92201b-1b37-4b10-b93a-e28315f2ba0e service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:54.984802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:54.987960 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:54.987960 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:54.987960 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:54.989006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:54.989986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:54.991963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.004923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1024/2020] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:55.010917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3a6ed-4e67-4b64-a3e5-0b81c9cc9ff0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.021659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3a6ed-4e67-4b64-a3e5-0b81c9cc9ff0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.028064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3692545-feab-4a4e-8a1a-022df5bc020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.028781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692545-feab-4a4e-8a1a-022df5bc020d admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.029074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692545-feab-4a4e-8a1a-022df5bc020d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.029357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692545-feab-4a4e-8a1a-022df5bc020d admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.030064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3692545-feab-4a4e-8a1a-022df5bc020d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.031973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3692545-feab-4a4e-8a1a-022df5bc020d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.130250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:55.139364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 997/2021] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:55 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:55.143151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.153381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ffb87-a48f-4ba8-a148-572728ece566 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.222273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1a720-efd0-4dea-8267-362d2ed12463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.222846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a720-efd0-4dea-8267-362d2ed12463 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.223078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a720-efd0-4dea-8267-362d2ed12463 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.223727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a720-efd0-4dea-8267-362d2ed12463 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.225451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1a720-efd0-4dea-8267-362d2ed12463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.228219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1a720-efd0-4dea-8267-362d2ed12463 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.303400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1025/2022] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:55 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:55.307570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3692545-feab-4a4e-8a1a-022df5bc020d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.318776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3692545-feab-4a4e-8a1a-022df5bc020d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.323409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bb3e6-638e-467d-874b-7fa4ae18af4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.324067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bb3e6-638e-467d-874b-7fa4ae18af4d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.324347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bb3e6-638e-467d-874b-7fa4ae18af4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.324650 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bb3e6-638e-467d-874b-7fa4ae18af4d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.325375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bb3e6-638e-467d-874b-7fa4ae18af4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.327010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bb3e6-638e-467d-874b-7fa4ae18af4d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.337833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 998/2023] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:55.341888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1a720-efd0-4dea-8267-362d2ed12463 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.352262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1a720-efd0-4dea-8267-362d2ed12463 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.357010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.357699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.359762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.359762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.359762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.360589 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.371111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1026/2024] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:55.375178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bb3e6-638e-467d-874b-7fa4ae18af4d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.385854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bb3e6-638e-467d-874b-7fa4ae18af4d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.390874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731bbe-95d8-48c1-8ef1-e7f45276f982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.391774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731bbe-95d8-48c1-8ef1-e7f45276f982 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.392080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731bbe-95d8-48c1-8ef1-e7f45276f982 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.392356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731bbe-95d8-48c1-8ef1-e7f45276f982 admin admin] PATH_INFO: `/v3/projects/5928f0aa1df044f78ace512911a04936/users/64d0e3be59b44781aa982d81511305a3/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.407286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731bbe-95d8-48c1-8ef1-e7f45276f982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5928f0aa1df044f78ace512911a04936, user_id=64d0e3be59b44781aa982d81511305a3, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.409996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731bbe-95d8-48c1-8ef1-e7f45276f982 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.415339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.417379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:55.431882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e09dd-f22e-4079-8170-abb491dc0a1a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.442358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.443229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.443538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.445475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] PATH_INFO: `/v3/projects/9e5f416610d34598b7ca363bf829a1f0` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.445475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5f416610d34598b7ca363bf829a1f0)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.447721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.522474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 999/2025] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:55 2022] PUT /identity/v3/projects/5928f0aa1df044f78ace512911a04936/users/64d0e3be59b44781aa982d81511305a3/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:55.532651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e71a7-c86e-463f-a9fe-7eb9cdccba60 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.532949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e71a7-c86e-463f-a9fe-7eb9cdccba60 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.533229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e71a7-c86e-463f-a9fe-7eb9cdccba60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.559286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-fe2e71a7-c86e-463f-a9fe-7eb9cdccba60 None None] MFA Rules not processed for user `64d0e3be59b44781aa982d81511305a3`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:55.560497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fe2e71a7-c86e-463f-a9fe-7eb9cdccba60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:55.604298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5f416610d34598b7ca363bf829a1f0', 'request_id': 'req-025767e7-83f7-4b8c-84bf-e27ba2a5017a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.630072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1000/2026] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:55.634603 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e71a7-c86e-463f-a9fe-7eb9cdccba60 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.653362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:55.682284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5f416610d34598b7ca363bf829a1f0', 'request_id': 'req-025767e7-83f7-4b8c-84bf-e27ba2a5017a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.688593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5f416610d34598b7ca363bf829a1f0', 'request_id': 'req-025767e7-83f7-4b8c-84bf-e27ba2a5017a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.698789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e71a7-c86e-463f-a9fe-7eb9cdccba60 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.701788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] The token cache is being invalidate because project 9e5f416610d34598b7ca363bf829a1f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:55.702081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.702811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.704328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1027/2027] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:55 2022] DELETE /identity/v3/projects/9e5f416610d34598b7ca363bf829a1f0 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:55.708259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.709689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:55.751247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:55.757894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.760819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.760819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.760819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.760819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.765292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.775628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025767e7-83f7-4b8c-84bf-e27ba2a5017a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.780364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.783706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.783706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.783706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] PATH_INFO: `/v3/users/6570c1936d5d4a68b7ab476da2153ddc` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.783706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6570c1936d5d4a68b7ab476da2153ddc)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.785392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.860365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1001/2028] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:55 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:55.864815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:55.866536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:55.929538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4eec13-9dad-48c0-87ab-f88577c0dcb8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:55.934144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:55.934809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:55.935094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:55.935375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:55.936061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:55.938012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:55.952936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6570c1936d5d4a68b7ab476da2153ddc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.965406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6570c1936d5d4a68b7ab476da2153ddc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.973666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6570c1936d5d4a68b7ab476da2153ddc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.974549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6570c1936d5d4a68b7ab476da2153ddc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:55.975228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6570c1936d5d4a68b7ab476da2153ddc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:56.007775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1002/2029] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:55 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:56.011878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:56.013529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:56.026793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7a8fc-2e4f-4b8f-bdb1-8c801ea7ecb1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:56.031926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1479e-4621-4848-98f3-c724ad25e9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:56.032121 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1479e-4621-4848-98f3-c724ad25e9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:56.032409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1479e-4621-4848-98f3-c724ad25e9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:56.032720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1479e-4621-4848-98f3-c724ad25e9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:56.033446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1479e-4621-4848-98f3-c724ad25e9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:56.035901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1479e-4621-4848-98f3-c724ad25e9b1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:56.046800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1003/2030] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:56.054188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1479e-4621-4848-98f3-c724ad25e9b1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:56.061087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1479e-4621-4848-98f3-c724ad25e9b1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:56.064981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1028/2031] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:55 2022] DELETE /identity/v3/users/6570c1936d5d4a68b7ab476da2153ddc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:56.068604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c4e51-092e-43a0-b48f-62cfaa71fd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:56.068604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c4e51-092e-43a0-b48f-62cfaa71fd1b admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:56.068604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c4e51-092e-43a0-b48f-62cfaa71fd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:56.068959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c4e51-092e-43a0-b48f-62cfaa71fd1b admin admin] PATH_INFO: `/v3/projects/3096c6350bc84a50bdc0cccce17aa75c/users/b030e73fa23f420ab34418c6583cd4ef/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:56.085387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c4e51-092e-43a0-b48f-62cfaa71fd1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3096c6350bc84a50bdc0cccce17aa75c, user_id=b030e73fa23f420ab34418c6583cd4ef, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:56.088036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c4e51-092e-43a0-b48f-62cfaa71fd1b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:56.156694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1004/2032] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:56 2022] PUT /identity/v3/projects/3096c6350bc84a50bdc0cccce17aa75c/users/b030e73fa23f420ab34418c6583cd4ef/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:56.161455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:56.218623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:56.224786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860908c2-97fb-49a5-8e7d-07539ed557b6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:56.286125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0349af3-1699-4bd4-a811-05dbce50dbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:56.287003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349af3-1699-4bd4-a811-05dbce50dbb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:56.287532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349af3-1699-4bd4-a811-05dbce50dbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:56.288550 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349af3-1699-4bd4-a811-05dbce50dbb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:56.289450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0349af3-1699-4bd4-a811-05dbce50dbb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:56.291563 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0349af3-1699-4bd4-a811-05dbce50dbb3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:56.302319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1029/2033] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:56.306428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c4e51-092e-43a0-b48f-62cfaa71fd1b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:56.314412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:56.319574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c4e51-092e-43a0-b48f-62cfaa71fd1b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:56.323987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:56.326366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:56.326366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:56.326366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] PATH_INFO: `/v3/projects/3096c6350bc84a50bdc0cccce17aa75c/users/b030e73fa23f420ab34418c6583cd4ef/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:56.327784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3096c6350bc84a50bdc0cccce17aa75c, user_id=b030e73fa23f420ab34418c6583cd4ef, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:56.330495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:56.351784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1005/2034] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:56 2022] PUT /identity/v3/projects/3096c6350bc84a50bdc0cccce17aa75c/users/b030e73fa23f420ab34418c6583cd4ef/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:56.355315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae282-a55d-4581-8091-b1d890c8f861 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:56.355685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae282-a55d-4581-8091-b1d890c8f861 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:56.355897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae282-a55d-4581-8091-b1d890c8f861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:56.391521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-02cae282-a55d-4581-8091-b1d890c8f861 None None] MFA Rules not processed for user `b030e73fa23f420ab34418c6583cd4ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:56.392804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-02cae282-a55d-4581-8091-b1d890c8f861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:56.467353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1030/2035] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:56.478488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:56.480211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:56.541746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbe4a5-86b8-44a0-b39e-a16617903ba7 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:56.545344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:56.545999 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:56.546285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:56.546570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:56.548586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:56.628475 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:56.629962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:56.639035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1006/2036] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:56.745725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cae282-a55d-4581-8091-b1d890c8f861 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:56.747250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-02cae282-a55d-4581-8091-b1d890c8f861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:56.846113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cae282-a55d-4581-8091-b1d890c8f861 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:56.853217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:56.854110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:56.854391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:56.854668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] PATH_INFO: `/v3/projects/deddebabc1f441cdaf2d6ba49ffd461c` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:56.855834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deddebabc1f441cdaf2d6ba49ffd461c)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:56.857811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:56.949574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:56.965678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deddebabc1f441cdaf2d6ba49ffd461c', 'request_id': 'req-5c786295-538d-46d8-8a3d-7768e7f8af39'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.059093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deddebabc1f441cdaf2d6ba49ffd461c', 'request_id': 'req-5c786295-538d-46d8-8a3d-7768e7f8af39'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.069920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deddebabc1f441cdaf2d6ba49ffd461c', 'request_id': 'req-5c786295-538d-46d8-8a3d-7768e7f8af39'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.086150 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] The token cache is being invalidate because project deddebabc1f441cdaf2d6ba49ffd461c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:57.086452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.087164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.088779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1031/2037] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:56 2022] DELETE /identity/v3/projects/deddebabc1f441cdaf2d6ba49ffd461c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:57.092686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:57.103871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca040b1-391b-4535-b7fa-32e3ceb67f46 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:57.166843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:57.168366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:57.168366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:57.168785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] PATH_INFO: `/v3/users/125b1e83c22c4ecab47428ee5b927fc4` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:57.173408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=125b1e83c22c4ecab47428ee5b927fc4)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:57.173408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:57.451655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125b1e83c22c4ecab47428ee5b927fc4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.452928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:57.460265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125b1e83c22c4ecab47428ee5b927fc4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.462716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125b1e83c22c4ecab47428ee5b927fc4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.462716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125b1e83c22c4ecab47428ee5b927fc4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.591051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ddc4e76e-e34d-40b2-81f1-bf0f7ae81576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125b1e83c22c4ecab47428ee5b927fc4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:57.610182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1007/2038] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:57 2022] DELETE /identity/v3/users/125b1e83c22c4ecab47428ee5b927fc4 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:58.066452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:58.067986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:58.084493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c786295-538d-46d8-8a3d-7768e7f8af39 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:58.145013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:58.145887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.146170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.146446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] PATH_INFO: `/v3/projects/561cec281d334770b041f277fd994841` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.147613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=561cec281d334770b041f277fd994841)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:58.149562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:58.263493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561cec281d334770b041f277fd994841', 'request_id': 'req-ad21d41e-aea4-4c9a-8f27-e031b61d0117'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:58.336571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561cec281d334770b041f277fd994841', 'request_id': 'req-ad21d41e-aea4-4c9a-8f27-e031b61d0117'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:58.342434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561cec281d334770b041f277fd994841', 'request_id': 'req-ad21d41e-aea4-4c9a-8f27-e031b61d0117'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:58.354041 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] The token cache is being invalidate because project 561cec281d334770b041f277fd994841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:58.354395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:58.355018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ad21d41e-aea4-4c9a-8f27-e031b61d0117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:58.356556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1032/2039] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:58 2022] DELETE /identity/v3/projects/561cec281d334770b041f277fd994841 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:58.361375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fed5b-ae2f-45cf-aff1-14235b3b280a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.361858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fed5b-ae2f-45cf-aff1-14235b3b280a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.361941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fed5b-ae2f-45cf-aff1-14235b3b280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.380203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-745fed5b-ae2f-45cf-aff1-14235b3b280a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:58.381509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-745fed5b-ae2f-45cf-aff1-14235b3b280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:58.442039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1008/2040] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:58.450002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8a28c-287e-46af-ad09-3cf2c809e374 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.450293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8a28c-287e-46af-ad09-3cf2c809e374 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.450568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8a28c-287e-46af-ad09-3cf2c809e374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.469622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a1d8a28c-287e-46af-ad09-3cf2c809e374 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:58.470812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a1d8a28c-287e-46af-ad09-3cf2c809e374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:58.479859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1033/2041] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:58.484018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fed5b-ae2f-45cf-aff1-14235b3b280a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:58.493904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fed5b-ae2f-45cf-aff1-14235b3b280a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:58.498498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:58.499157 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.499444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.499721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.500451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:58.500939 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:58.502557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:58.517451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1009/2042] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:58.519026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8a28c-287e-46af-ad09-3cf2c809e374 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:58.528636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8a28c-287e-46af-ad09-3cf2c809e374 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:58.533772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5333aa5-ea34-4165-a6d5-ab4da6a93c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:58.535041 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5333aa5-ea34-4165-a6d5-ab4da6a93c97 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.535394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5333aa5-ea34-4165-a6d5-ab4da6a93c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.535758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5333aa5-ea34-4165-a6d5-ab4da6a93c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.536543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5333aa5-ea34-4165-a6d5-ab4da6a93c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:58.538521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5333aa5-ea34-4165-a6d5-ab4da6a93c97 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:58.619038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1034/2043] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:58 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:58.622912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:58.633824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5922ba-0aed-43c6-9aaf-909d56e41c8e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:58.688087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ce97d-db05-4273-a280-bbca0aa939e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:58.688855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ce97d-db05-4273-a280-bbca0aa939e1 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.689409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ce97d-db05-4273-a280-bbca0aa939e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.689691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ce97d-db05-4273-a280-bbca0aa939e1 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.690886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ce97d-db05-4273-a280-bbca0aa939e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:58.692820 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ce97d-db05-4273-a280-bbca0aa939e1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:58.767157 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1010/2044] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:58 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:58.771735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5333aa5-ea34-4165-a6d5-ab4da6a93c97 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:58.782295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5333aa5-ea34-4165-a6d5-ab4da6a93c97 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:58.787261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c95558-3a57-4af9-8d7a-c0822519e37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:58.787921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c95558-3a57-4af9-8d7a-c0822519e37a admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.788208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c95558-3a57-4af9-8d7a-c0822519e37a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.788490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c95558-3a57-4af9-8d7a-c0822519e37a admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.789280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c95558-3a57-4af9-8d7a-c0822519e37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:58.790893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c95558-3a57-4af9-8d7a-c0822519e37a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:58.802457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1035/2045] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:58.806389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ce97d-db05-4273-a280-bbca0aa939e1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:58.817173 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ce97d-db05-4273-a280-bbca0aa939e1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:58.821923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1abd1-c63d-47e4-8613-72558b12a687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:58.822573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1abd1-c63d-47e4-8613-72558b12a687 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.822860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1abd1-c63d-47e4-8613-72558b12a687 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.823141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1abd1-c63d-47e4-8613-72558b12a687 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.823867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1abd1-c63d-47e4-8613-72558b12a687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:58.825471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1abd1-c63d-47e4-8613-72558b12a687 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:58.837643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1011/2046] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:58.841413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c95558-3a57-4af9-8d7a-c0822519e37a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:58.847638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:58.851291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c95558-3a57-4af9-8d7a-c0822519e37a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:58.855845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:58.856743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.857026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.857302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] PATH_INFO: `/v3/projects/a8d7918c87fc4feab324728c4739d326/users/881f9aedfd644810ab368bae004716c6/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.871587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d7918c87fc4feab324728c4739d326, user_id=881f9aedfd644810ab368bae004716c6, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:58.874246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:58.934129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1036/2047] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:58 2022] PUT /identity/v3/projects/a8d7918c87fc4feab324728c4739d326/users/881f9aedfd644810ab368bae004716c6/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:58.951158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eea65-ac65-426b-9602-ac243518b0de None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:58.951553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eea65-ac65-426b-9602-ac243518b0de None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:58.951923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eea65-ac65-426b-9602-ac243518b0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:58.980141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8b8eea65-ac65-426b-9602-ac243518b0de None None] MFA Rules not processed for user `881f9aedfd644810ab368bae004716c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:58.981462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8b8eea65-ac65-426b-9602-ac243518b0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:59.001449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:59.046929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1012/2048] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:35:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:59.058058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:59.059765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:59.122537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac3db-7d01-4e46-b14d-31fd22dcc9b6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:59.126133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:59.126784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:59.127098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:59.127382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:59.137367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:59.139154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:59.148168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1037/2049] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:02.025106 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900e39a-7f53-44cb-9d67-ecf3b27393cb None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:02.025551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900e39a-7f53-44cb-9d67-ecf3b27393cb None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:02.025766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900e39a-7f53-44cb-9d67-ecf3b27393cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:02.042716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-3900e39a-7f53-44cb-9d67-ecf3b27393cb None None] MFA Rules not processed for user `881f9aedfd644810ab368bae004716c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:02.043923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3900e39a-7f53-44cb-9d67-ecf3b27393cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:02.051858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1013/2050] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:05.852946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:05.854504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:05.929613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499d967-a932-41c0-ac2b-6f04ccd11912 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:05.936064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:05.937093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:05.937580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:05.937803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] PATH_INFO: `/v3/users/542b5913990f40c5adc90d4685f559ec` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:05.939400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] RBAC: Authorizing `identity:delete_user(user_id=542b5913990f40c5adc90d4685f559ec)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:05.941516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:06.166486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542b5913990f40c5adc90d4685f559ec'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:06.182408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542b5913990f40c5adc90d4685f559ec'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:06.190237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542b5913990f40c5adc90d4685f559ec'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:06.191091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542b5913990f40c5adc90d4685f559ec'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:06.191716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542b5913990f40c5adc90d4685f559ec'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:06.303237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1038/2051] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:05 2022] DELETE /identity/v3/users/542b5913990f40c5adc90d4685f559ec => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:06.307802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02171b14-c0a9-456d-9e33-359eab0b650e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:06.308286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02171b14-c0a9-456d-9e33-359eab0b650e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:06.308381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02171b14-c0a9-456d-9e33-359eab0b650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:06.326965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-02171b14-c0a9-456d-9e33-359eab0b650e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:06.328442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-02171b14-c0a9-456d-9e33-359eab0b650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:06.390024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1014/2052] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:06.400488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:06.402201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:06.408349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:06.473339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651cf77-de86-4274-b569-bbe3f3a9026c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:06.477293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:06.477930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:06.478220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:06.478496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:06.487826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:06.489325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:06.499243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1039/2053] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:06.855510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02171b14-c0a9-456d-9e33-359eab0b650e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:06.858373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-02171b14-c0a9-456d-9e33-359eab0b650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:06.925238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02171b14-c0a9-456d-9e33-359eab0b650e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:06.929400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:06.930178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:06.930480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:06.930763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:06.933071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:07.005301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:07.007072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:07.009985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:07.010212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:07.022125 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1015/2054] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:07.052358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17255ad8-a5b7-47c6-8d98-4e8c3001a8b1 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:07.067425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:07.068778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:07.069064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:07.069348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] PATH_INFO: `/v3/projects/0bfae81d2de24f5bb456de555da2e229` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:07.070843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bfae81d2de24f5bb456de555da2e229)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:07.072494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:07.183658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bfae81d2de24f5bb456de555da2e229', 'request_id': 'req-944a951f-1c1e-41e8-b9fa-1cee66d0c797'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.270601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bfae81d2de24f5bb456de555da2e229', 'request_id': 'req-944a951f-1c1e-41e8-b9fa-1cee66d0c797'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.295865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bfae81d2de24f5bb456de555da2e229', 'request_id': 'req-944a951f-1c1e-41e8-b9fa-1cee66d0c797'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.335630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] The token cache is being invalidate because project 0bfae81d2de24f5bb456de555da2e229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:07.336349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.337470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.339419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1040/2055] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:07 2022] DELETE /identity/v3/projects/0bfae81d2de24f5bb456de555da2e229 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:07.344225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:07.353932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:07.360078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7803438-a2ca-468a-8929-845f8b60051b service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:07.453405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:07.454312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:07.454604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:07.454894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] PATH_INFO: `/v3/users/faaf870aee544eddb9a079bceff6f9a2` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:07.457466 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] RBAC: Authorizing `identity:delete_user(user_id=faaf870aee544eddb9a079bceff6f9a2)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:07.460119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:07.751946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faaf870aee544eddb9a079bceff6f9a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.761344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faaf870aee544eddb9a079bceff6f9a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.762885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faaf870aee544eddb9a079bceff6f9a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.763717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faaf870aee544eddb9a079bceff6f9a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.836225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faaf870aee544eddb9a079bceff6f9a2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:07.866253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1016/2056] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:07 2022] DELETE /identity/v3/users/faaf870aee544eddb9a079bceff6f9a2 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:08.592697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:08.595348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:08.614887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a951f-1c1e-41e8-b9fa-1cee66d0c797 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:08.736893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:08.747611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:08.747893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:08.749505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] PATH_INFO: `/v3/projects/1c5d9fca2e5748c5b6b95a9203c9523b` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:08.751213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5d9fca2e5748c5b6b95a9203c9523b)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:08.755378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:08.906703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:08.909302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:08.930203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:08.940060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066e58a-c39f-4de7-b531-467593b9018b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:08.959115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5d9fca2e5748c5b6b95a9203c9523b', 'request_id': 'req-a3c049df-ec4e-493e-9ce0-60d09d027334'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.070828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5d9fca2e5748c5b6b95a9203c9523b', 'request_id': 'req-a3c049df-ec4e-493e-9ce0-60d09d027334'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.078434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5d9fca2e5748c5b6b95a9203c9523b', 'request_id': 'req-a3c049df-ec4e-493e-9ce0-60d09d027334'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.101098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] The token cache is being invalidate because project 1c5d9fca2e5748c5b6b95a9203c9523b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:09.102117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.103642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a3c049df-ec4e-493e-9ce0-60d09d027334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.105316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:09.106294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.106841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.107216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] PATH_INFO: `/v3/users/a389c2af5f214afebbf87748a31e1dae` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.108728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1041/2057] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:08 2022] DELETE /identity/v3/projects/1c5d9fca2e5748c5b6b95a9203c9523b => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:09.114350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a389c2af5f214afebbf87748a31e1dae)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:09.114867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37767fc2-cca0-4451-bdff-5a55b6295440 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.117507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:09.140836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37767fc2-cca0-4451-bdff-5a55b6295440 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.140836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37767fc2-cca0-4451-bdff-5a55b6295440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.167522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-37767fc2-cca0-4451-bdff-5a55b6295440 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:09.169079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-37767fc2-cca0-4451-bdff-5a55b6295440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:09.180190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1042/2058] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:09.187484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36cfe-21c8-4c1d-9d02-9e1164dbe1f5 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.189592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36cfe-21c8-4c1d-9d02-9e1164dbe1f5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.189890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36cfe-21c8-4c1d-9d02-9e1164dbe1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.218042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-06d36cfe-21c8-4c1d-9d02-9e1164dbe1f5 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:09.219285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-06d36cfe-21c8-4c1d-9d02-9e1164dbe1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:09.226200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:09.228674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1043/2059] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:09.232688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36cfe-21c8-4c1d-9d02-9e1164dbe1f5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:09.243157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36cfe-21c8-4c1d-9d02-9e1164dbe1f5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:09.248701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:09.249651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.249905 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.250315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.251152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:09.251611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:09.253916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:09.272598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1044/2060] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:36:09.276117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:09.288727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b5089-3c9a-4d7d-bbd3-e68156771e67 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:09.293639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:09.294370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.294722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.295068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.295868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:09.297857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:09.331482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a389c2af5f214afebbf87748a31e1dae'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.342786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a389c2af5f214afebbf87748a31e1dae'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.343759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a389c2af5f214afebbf87748a31e1dae'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.344676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a389c2af5f214afebbf87748a31e1dae'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.414548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-68fc2303-df67-4088-bf0a-85821e39bed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a389c2af5f214afebbf87748a31e1dae'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:09.415228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1045/2061] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:09 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:09.436630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:09.439800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:09.461243 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1017/2062] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:08 2022] DELETE /identity/v3/users/a389c2af5f214afebbf87748a31e1dae => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:09.465709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc040fc-1319-4246-a6e6-0f3d9ce19091 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.466178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc040fc-1319-4246-a6e6-0f3d9ce19091 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.466468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc040fc-1319-4246-a6e6-0f3d9ce19091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.497551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ccc040fc-1319-4246-a6e6-0f3d9ce19091 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:09.500614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ccc040fc-1319-4246-a6e6-0f3d9ce19091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:09.652222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adbf33-370c-46f5-b974-eccc4f374226 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:09.654332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1018/2063] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:09.658446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a42c3-f850-424a-a030-7897832fcd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:09.659986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a42c3-f850-424a-a030-7897832fcd40 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.660461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a42c3-f850-424a-a030-7897832fcd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.660873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a42c3-f850-424a-a030-7897832fcd40 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.661138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc040fc-1319-4246-a6e6-0f3d9ce19091 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:09.661699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a42c3-f850-424a-a030-7897832fcd40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:09.662682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ccc040fc-1319-4246-a6e6-0f3d9ce19091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:09.663868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a42c3-f850-424a-a030-7897832fcd40 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:09.742011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc040fc-1319-4246-a6e6-0f3d9ce19091 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:09.745851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2890c21-4486-4e68-8ca7-5acaa64e39d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:09.746744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2890c21-4486-4e68-8ca7-5acaa64e39d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.747118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2890c21-4486-4e68-8ca7-5acaa64e39d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.747493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2890c21-4486-4e68-8ca7-5acaa64e39d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.758832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2890c21-4486-4e68-8ca7-5acaa64e39d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:09.760356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2890c21-4486-4e68-8ca7-5acaa64e39d9 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:09.771711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1019/2064] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:36:09.802323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1046/2065] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:09 2022] POST /identity/v3/users => generated 283 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:09.803638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2890c21-4486-4e68-8ca7-5acaa64e39d9 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:09.826992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2890c21-4486-4e68-8ca7-5acaa64e39d9 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:09.832404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002046f8-fd00-4ea5-b058-52e26d159528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:09.835017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002046f8-fd00-4ea5-b058-52e26d159528 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.835514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002046f8-fd00-4ea5-b058-52e26d159528 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.835910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002046f8-fd00-4ea5-b058-52e26d159528 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.836774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002046f8-fd00-4ea5-b058-52e26d159528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:09.838617 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002046f8-fd00-4ea5-b058-52e26d159528 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:09.854399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1020/2066] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:09.855348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a42c3-f850-424a-a030-7897832fcd40 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:09.875022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a42c3-f850-424a-a030-7897832fcd40 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:09.881893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9b70b-4c04-44b8-aa7b-ef128f66f3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:09.882721 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b70b-4c04-44b8-aa7b-ef128f66f3cf admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.884323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b70b-4c04-44b8-aa7b-ef128f66f3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.884323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b70b-4c04-44b8-aa7b-ef128f66f3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.884323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9b70b-4c04-44b8-aa7b-ef128f66f3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:09.885924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9b70b-4c04-44b8-aa7b-ef128f66f3cf admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:09.899863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1047/2067] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:09.904192 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002046f8-fd00-4ea5-b058-52e26d159528 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:09.922224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002046f8-fd00-4ea5-b058-52e26d159528 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:09.929479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49dc1-c523-4edf-9f3a-9a222a272ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:09.930569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49dc1-c523-4edf-9f3a-9a222a272ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:09.930860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49dc1-c523-4edf-9f3a-9a222a272ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:09.931187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49dc1-c523-4edf-9f3a-9a222a272ef5 admin admin] PATH_INFO: `/v3/projects/609e77dfef414038b32f01bb25d177ee/users/8439c8343b754ae98ac0788b05cfe0e2/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:09.948448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def49dc1-c523-4edf-9f3a-9a222a272ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609e77dfef414038b32f01bb25d177ee, user_id=8439c8343b754ae98ac0788b05cfe0e2, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:09.951112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def49dc1-c523-4edf-9f3a-9a222a272ef5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:10.082237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1021/2068] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:09 2022] PUT /identity/v3/projects/609e77dfef414038b32f01bb25d177ee/users/8439c8343b754ae98ac0788b05cfe0e2/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:10.094504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af63e-de85-4cac-80ab-6ace11c2ad79 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:10.094808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af63e-de85-4cac-80ab-6ace11c2ad79 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:10.095089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af63e-de85-4cac-80ab-6ace11c2ad79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:10.134939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-637af63e-de85-4cac-80ab-6ace11c2ad79 None None] MFA Rules not processed for user `8439c8343b754ae98ac0788b05cfe0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:10.136820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-637af63e-de85-4cac-80ab-6ace11c2ad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:10.201671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:10.239568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1048/2069] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:10.244346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49dc1-c523-4edf-9f3a-9a222a272ef5 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:10.260044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:10.335295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49dc1-c523-4edf-9f3a-9a222a272ef5 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:10.342368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2b589-e60d-4522-8e33-391cdefa978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:10.343055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b589-e60d-4522-8e33-391cdefa978f admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:10.343508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b589-e60d-4522-8e33-391cdefa978f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:10.343950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b589-e60d-4522-8e33-391cdefa978f admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:10.344722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2b589-e60d-4522-8e33-391cdefa978f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:10.346886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2b589-e60d-4522-8e33-391cdefa978f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:10.452624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af63e-de85-4cac-80ab-6ace11c2ad79 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:10.454041 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-637af63e-de85-4cac-80ab-6ace11c2ad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:10.469263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af63e-de85-4cac-80ab-6ace11c2ad79 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:10.474860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:10.475747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:10.476273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:10.477295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] PATH_INFO: `/v3/users/704b1cb48c7249d69ab69172d1a157b1` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:10.479120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=704b1cb48c7249d69ab69172d1a157b1)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:10.481261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:10.487211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1022/2070] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:10 2022] POST /identity/v3/projects => generated 365 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:10.488393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2b589-e60d-4522-8e33-391cdefa978f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:10.562042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2b589-e60d-4522-8e33-391cdefa978f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:10.567621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:10.568297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:10.568624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:10.568905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:10.569599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:10.571546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:10.679045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1023/2071] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:10 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:10.679045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:10.679045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:10.692514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0f83a-180f-4669-9d78-299a0c0543d4 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:10.697490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:10.698413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:10.698703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:10.698987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] PATH_INFO: `/v3/projects/c143dfe0be6846548dbdd4040a1f2515` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:10.700253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c143dfe0be6846548dbdd4040a1f2515)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:10.702183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:10.761970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '704b1cb48c7249d69ab69172d1a157b1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.774449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '704b1cb48c7249d69ab69172d1a157b1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.781580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '704b1cb48c7249d69ab69172d1a157b1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.782268 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '704b1cb48c7249d69ab69172d1a157b1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.782871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '704b1cb48c7249d69ab69172d1a157b1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.847744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c143dfe0be6846548dbdd4040a1f2515', 'request_id': 'req-c430401d-1c56-44bd-8129-79eb71670f0b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.862788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1049/2072] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:10 2022] DELETE /identity/v3/users/704b1cb48c7249d69ab69172d1a157b1 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:10.863523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:10.864919 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:10.947432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e811296-5e8e-4625-b7db-0e517ee193c4 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:10.952431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3b17c-2494-4f80-80ba-2ff4202a904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:10.953039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3b17c-2494-4f80-80ba-2ff4202a904e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:10.953573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3b17c-2494-4f80-80ba-2ff4202a904e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:10.953690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3b17c-2494-4f80-80ba-2ff4202a904e admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:10.954175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c143dfe0be6846548dbdd4040a1f2515', 'request_id': 'req-c430401d-1c56-44bd-8129-79eb71670f0b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.954609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3b17c-2494-4f80-80ba-2ff4202a904e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:10.956681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3b17c-2494-4f80-80ba-2ff4202a904e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:10.963397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c143dfe0be6846548dbdd4040a1f2515', 'request_id': 'req-c430401d-1c56-44bd-8129-79eb71670f0b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.971006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1050/2073] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:10.976021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3b17c-2494-4f80-80ba-2ff4202a904e admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:10.977757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] The token cache is being invalidate because project c143dfe0be6846548dbdd4040a1f2515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:10.980643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.980643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c430401d-1c56-44bd-8129-79eb71670f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:10.981638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1024/2074] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:10 2022] DELETE /identity/v3/projects/c143dfe0be6846548dbdd4040a1f2515 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:10.989894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9bb19-6c25-4ef8-b2f6-026b3cf9fc91 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:10.990334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9bb19-6c25-4ef8-b2f6-026b3cf9fc91 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:10.990623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9bb19-6c25-4ef8-b2f6-026b3cf9fc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:10.990724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3b17c-2494-4f80-80ba-2ff4202a904e admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:10.995549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:10.996579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:10.996798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:10.997080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] PATH_INFO: `/v3/projects/6c165280101947d8a0190f790c316324/users/fdf36f0fb3004f73ba18018e2dbc6e94/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.009720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-26a9bb19-6c25-4ef8-b2f6-026b3cf9fc91 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:11.011102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-26a9bb19-6c25-4ef8-b2f6-026b3cf9fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.013680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c165280101947d8a0190f790c316324, user_id=fdf36f0fb3004f73ba18018e2dbc6e94, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.016324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.020402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1025/2075] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:11.028155 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93897c-7e36-40ad-865d-52dac5437a37 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.028573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93897c-7e36-40ad-865d-52dac5437a37 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.028863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93897c-7e36-40ad-865d-52dac5437a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.047377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-af93897c-7e36-40ad-865d-52dac5437a37 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:11.048576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-af93897c-7e36-40ad-865d-52dac5437a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.056488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1026/2076] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:11.060389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93897c-7e36-40ad-865d-52dac5437a37 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.069378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93897c-7e36-40ad-865d-52dac5437a37 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.074622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.075237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.075517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.075793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.076502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:11.076838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.078394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.090187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1027/2077] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:36:11.094055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.104077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7666ffa-5cfb-4ad0-b6a6-e07e42d4e261 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.108715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590b343-e3ac-4335-9b15-e053dd753e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.109598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590b343-e3ac-4335-9b15-e053dd753e93 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.109916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590b343-e3ac-4335-9b15-e053dd753e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.110189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590b343-e3ac-4335-9b15-e053dd753e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.115768 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590b343-e3ac-4335-9b15-e053dd753e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.115768 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590b343-e3ac-4335-9b15-e053dd753e93 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.115978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1051/2078] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:10 2022] PUT /identity/v3/projects/6c165280101947d8a0190f790c316324/users/fdf36f0fb3004f73ba18018e2dbc6e94/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:11.119527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.121488 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.213395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:11.236738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8ccce-f8f6-400b-8812-899c571aceca admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.241560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.242284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.242573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.242881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.243615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.246111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.259970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1052/2079] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:11.261108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.278213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.283071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1028/2080] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:11 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:11.284597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590b343-e3ac-4335-9b15-e053dd753e93 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.297046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f72e9-cf02-41ae-bb1b-7bc16765bcec admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.309457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:11.364737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590b343-e3ac-4335-9b15-e053dd753e93 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.371354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b1db-0b36-4511-935f-98165921c308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.372347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b1db-0b36-4511-935f-98165921c308 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.372602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b1db-0b36-4511-935f-98165921c308 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.372887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b1db-0b36-4511-935f-98165921c308 admin admin] PATH_INFO: `/v3/projects/6c165280101947d8a0190f790c316324/users/fdf36f0fb3004f73ba18018e2dbc6e94/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.374921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088b1db-0b36-4511-935f-98165921c308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c165280101947d8a0190f790c316324, user_id=fdf36f0fb3004f73ba18018e2dbc6e94, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.378489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088b1db-0b36-4511-935f-98165921c308 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.386036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.387115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.387520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.387899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] PATH_INFO: `/v3/projects/2d9a178dbd5e430f874fa216367424dc` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.389421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9a178dbd5e430f874fa216367424dc)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.391393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1029/2081] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:11 2022] PUT /identity/v3/projects/6c165280101947d8a0190f790c316324/users/fdf36f0fb3004f73ba18018e2dbc6e94/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:11.391889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.392298 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b1db-0b36-4511-935f-98165921c308 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.464665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b1db-0b36-4511-935f-98165921c308 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.469668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1b2a3-6eff-48d8-84c7-76ded2bb3085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.470413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1b2a3-6eff-48d8-84c7-76ded2bb3085 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.470769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1b2a3-6eff-48d8-84c7-76ded2bb3085 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.471278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1b2a3-6eff-48d8-84c7-76ded2bb3085 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.472123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1b2a3-6eff-48d8-84c7-76ded2bb3085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.474307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1b2a3-6eff-48d8-84c7-76ded2bb3085 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.571120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9a178dbd5e430f874fa216367424dc', 'request_id': 'req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:11.602092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1030/2082] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:11 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:11.603388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e9ded-dab3-4d23-b74e-cee8832b9b60 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.603684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e9ded-dab3-4d23-b74e-cee8832b9b60 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.603966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e9ded-dab3-4d23-b74e-cee8832b9b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.632011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c82e9ded-dab3-4d23-b74e-cee8832b9b60 None None] MFA Rules not processed for user `fdf36f0fb3004f73ba18018e2dbc6e94`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:11.633248 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c82e9ded-dab3-4d23-b74e-cee8832b9b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.685053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9a178dbd5e430f874fa216367424dc', 'request_id': 'req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:11.692306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9a178dbd5e430f874fa216367424dc', 'request_id': 'req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:11.699301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1031/2083] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:11.700245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e9ded-dab3-4d23-b74e-cee8832b9b60 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.704681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] The token cache is being invalidate because project 2d9a178dbd5e430f874fa216367424dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:11.705506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:11.706176 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:11.708078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1053/2084] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:11 2022] DELETE /identity/v3/projects/2d9a178dbd5e430f874fa216367424dc => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:11.712052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.713783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.762267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e9ded-dab3-4d23-b74e-cee8832b9b60 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.766860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e85251-1458-42ce-82cb-a970a309c82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.767525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85251-1458-42ce-82cb-a970a309c82e admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.767811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85251-1458-42ce-82cb-a970a309c82e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.768114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85251-1458-42ce-82cb-a970a309c82e admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.768888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e85251-1458-42ce-82cb-a970a309c82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.770497 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e85251-1458-42ce-82cb-a970a309c82e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.770604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff4a1-d9e2-4e09-a87b-2e3f8331c788 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.774206 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.774848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.775128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.775505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.777366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.780353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1032/2085] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:11.781736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78381b-2928-469c-8e7d-7e2d9006a5dd None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.782026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78381b-2928-469c-8e7d-7e2d9006a5dd None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.782304 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78381b-2928-469c-8e7d-7e2d9006a5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.792440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.793953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.801039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7a78381b-2928-469c-8e7d-7e2d9006a5dd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:11.802234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7a78381b-2928-469c-8e7d-7e2d9006a5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.804393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1054/2086] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:11.805066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.806757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.811139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1033/2087] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:11.820464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534a04-bef2-4147-9eee-137a0b5241ef None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.820970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534a04-bef2-4147-9eee-137a0b5241ef None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.821317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534a04-bef2-4147-9eee-137a0b5241ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.823542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2509058-eb52-4051-b0c6-d8dbe424703c service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.828542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce317fc-b314-4162-8a52-86148400e6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.829215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce317fc-b314-4162-8a52-86148400e6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.829498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce317fc-b314-4162-8a52-86148400e6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.829879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce317fc-b314-4162-8a52-86148400e6b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.830628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce317fc-b314-4162-8a52-86148400e6b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.832257 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce317fc-b314-4162-8a52-86148400e6b3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.840128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-27534a04-bef2-4147-9eee-137a0b5241ef None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:11.841535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-27534a04-bef2-4147-9eee-137a0b5241ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:11.843487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1055/2088] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:11.847742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce317fc-b314-4162-8a52-86148400e6b3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.850971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1034/2089] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:11.854937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27534a04-bef2-4147-9eee-137a0b5241ef None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.858102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce317fc-b314-4162-8a52-86148400e6b3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.863631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577399a7-0115-4c70-a569-b18e84ea7f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.864538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577399a7-0115-4c70-a569-b18e84ea7f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.864674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27534a04-bef2-4147-9eee-137a0b5241ef None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.864839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577399a7-0115-4c70-a569-b18e84ea7f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.865110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577399a7-0115-4c70-a569-b18e84ea7f1f admin admin] PATH_INFO: `/v3/projects/1601ada8906247f2bb4bd21f78517042/users/f8478d03a85d4c7b8c7d777ac4969b21/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.869624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.870278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.870566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.870838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.871625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:11.871938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.873723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.882727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577399a7-0115-4c70-a569-b18e84ea7f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1601ada8906247f2bb4bd21f78517042, user_id=f8478d03a85d4c7b8c7d777ac4969b21, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.885994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577399a7-0115-4c70-a569-b18e84ea7f1f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.887562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1035/2090] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:36:11.891541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:11.901925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ece12-c054-48f3-88c2-8bdefe179d4b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:11.907663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd7ab0-cb67-4810-be08-cbe90abb6ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:11.908316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd7ab0-cb67-4810-be08-cbe90abb6ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.908632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd7ab0-cb67-4810-be08-cbe90abb6ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.908908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd7ab0-cb67-4810-be08-cbe90abb6ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.910199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd7ab0-cb67-4810-be08-cbe90abb6ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:11.911520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd7ab0-cb67-4810-be08-cbe90abb6ce4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:11.962776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1056/2091] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:11 2022] PUT /identity/v3/projects/1601ada8906247f2bb4bd21f78517042/users/f8478d03a85d4c7b8c7d777ac4969b21/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:11.964318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0c93-017d-455c-91f4-919fc40bf30c None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:11.964654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0c93-017d-455c-91f4-919fc40bf30c None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:11.964942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0c93-017d-455c-91f4-919fc40bf30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:11.992296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-39fc0c93-017d-455c-91f4-919fc40bf30c None None] MFA Rules not processed for user `f8478d03a85d4c7b8c7d777ac4969b21`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:12.001500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-39fc0c93-017d-455c-91f4-919fc40bf30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:12.031405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1036/2092] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:11 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:12.036105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd7ab0-cb67-4810-be08-cbe90abb6ce4 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:12.070358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1057/2093] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:12.082696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0c93-017d-455c-91f4-919fc40bf30c None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:12.110225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd7ab0-cb67-4810-be08-cbe90abb6ce4 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:12.115145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6af2-6b51-4ba4-a7f1-4d5e91f6ceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:12.115817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6af2-6b51-4ba4-a7f1-4d5e91f6ceda admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:12.116129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6af2-6b51-4ba4-a7f1-4d5e91f6ceda admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:12.116454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6af2-6b51-4ba4-a7f1-4d5e91f6ceda admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:12.117143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b6af2-6b51-4ba4-a7f1-4d5e91f6ceda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:12.119074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b6af2-6b51-4ba4-a7f1-4d5e91f6ceda admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:12.141341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0c93-017d-455c-91f4-919fc40bf30c None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:12.159596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:12.193304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771be235-77e7-4fdc-ac8e-97ab24970e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:12.193952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771be235-77e7-4fdc-ac8e-97ab24970e98 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:12.194305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771be235-77e7-4fdc-ac8e-97ab24970e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:12.194514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771be235-77e7-4fdc-ac8e-97ab24970e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:12.214758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1037/2094] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:12 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:12.216090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6af2-6b51-4ba4-a7f1-4d5e91f6ceda admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:12.225850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6af2-6b51-4ba4-a7f1-4d5e91f6ceda admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:12.230518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb67ff4-f2b7-492f-bd11-4d3e0fd594a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:12.231161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb67ff4-f2b7-492f-bd11-4d3e0fd594a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:12.231450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb67ff4-f2b7-492f-bd11-4d3e0fd594a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:12.231782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb67ff4-f2b7-492f-bd11-4d3e0fd594a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:12.242087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb67ff4-f2b7-492f-bd11-4d3e0fd594a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:12.244451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb67ff4-f2b7-492f-bd11-4d3e0fd594a1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:12.256898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1038/2095] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:12.262403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771be235-77e7-4fdc-ac8e-97ab24970e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:12.262403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771be235-77e7-4fdc-ac8e-97ab24970e98 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:12.263449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb67ff4-f2b7-492f-bd11-4d3e0fd594a1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:12.271352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1058/2096] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:36:12.274144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb67ff4-f2b7-492f-bd11-4d3e0fd594a1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:12.279245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdce055-b712-4ebf-8270-5f79dab20670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:12.280677 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdce055-b712-4ebf-8270-5f79dab20670 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:12.280951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdce055-b712-4ebf-8270-5f79dab20670 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:12.281229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdce055-b712-4ebf-8270-5f79dab20670 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:12.281991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdce055-b712-4ebf-8270-5f79dab20670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:12.283962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdce055-b712-4ebf-8270-5f79dab20670 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:12.294561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1039/2097] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:12.299928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771be235-77e7-4fdc-ac8e-97ab24970e98 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:12.308833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:12.312730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771be235-77e7-4fdc-ac8e-97ab24970e98 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:12.318328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f67ce5-6dc3-4734-9395-b74f38a666ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:12.319254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f67ce5-6dc3-4734-9395-b74f38a666ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:12.319546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f67ce5-6dc3-4734-9395-b74f38a666ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:12.319830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f67ce5-6dc3-4734-9395-b74f38a666ec admin admin] PATH_INFO: `/v3/projects/84b1f407cf0e45938d092cb59207f3e3/users/0d96ae2c0a894472a17fd24dc394a385/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:12.342613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f67ce5-6dc3-4734-9395-b74f38a666ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b1f407cf0e45938d092cb59207f3e3, user_id=0d96ae2c0a894472a17fd24dc394a385, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:12.345457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f67ce5-6dc3-4734-9395-b74f38a666ec admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:12.426631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1059/2098] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:12 2022] PUT /identity/v3/projects/84b1f407cf0e45938d092cb59207f3e3/users/0d96ae2c0a894472a17fd24dc394a385/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:12.438264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab876fa-94a8-4a8c-ad0a-9919d9f3a3c3 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:12.438607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab876fa-94a8-4a8c-ad0a-9919d9f3a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:12.438894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab876fa-94a8-4a8c-ad0a-9919d9f3a3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:12.519145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-1ab876fa-94a8-4a8c-ad0a-9919d9f3a3c3 None None] MFA Rules not processed for user `0d96ae2c0a894472a17fd24dc394a385`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:12.520843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1ab876fa-94a8-4a8c-ad0a-9919d9f3a3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:12.619304 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1040/2099] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:12.624693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f67ce5-6dc3-4734-9395-b74f38a666ec admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:12.712097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f67ce5-6dc3-4734-9395-b74f38a666ec admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:12.714760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:12.715416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:12.715841 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:12.716129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:12.716873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:12.719626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:12.848841 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1060/2100] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:12 2022] POST /identity/v3/projects => generated 361 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:12.852848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab876fa-94a8-4a8c-ad0a-9919d9f3a3c3 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:12.863436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab876fa-94a8-4a8c-ad0a-9919d9f3a3c3 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:12.895740 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:12.927491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df8bc7-6102-4cd9-aeb9-7e3d6449c3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:12.929494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df8bc7-6102-4cd9-aeb9-7e3d6449c3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:12.929494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df8bc7-6102-4cd9-aeb9-7e3d6449c3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:12.929494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df8bc7-6102-4cd9-aeb9-7e3d6449c3f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:12.929494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df8bc7-6102-4cd9-aeb9-7e3d6449c3f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:12.931531 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df8bc7-6102-4cd9-aeb9-7e3d6449c3f0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.011154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.011391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1041/2101] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:12 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:13.014883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df8bc7-6102-4cd9-aeb9-7e3d6449c3f0 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.015288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:13.025540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df8bc7-6102-4cd9-aeb9-7e3d6449c3f0 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.028370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaac622-8094-4516-b6ff-271eb2e8bdf8 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6c462-5eae-4f58-874d-9169db396b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.030858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6c462-5eae-4f58-874d-9169db396b40 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.031146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6c462-5eae-4f58-874d-9169db396b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.031421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6c462-5eae-4f58-874d-9169db396b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.032159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6c462-5eae-4f58-874d-9169db396b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.033043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.033783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6c462-5eae-4f58-874d-9169db396b40 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.033900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.034196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.034516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] PATH_INFO: `/v3/users/8439c8343b754ae98ac0788b05cfe0e2` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.036166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8439c8343b754ae98ac0788b05cfe0e2)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.038099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.047383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1042/2102] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:13.048448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6c462-5eae-4f58-874d-9169db396b40 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.058838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6c462-5eae-4f58-874d-9169db396b40 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.063563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983212c-01aa-40c3-afd5-dc4c3ed92fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.064724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983212c-01aa-40c3-afd5-dc4c3ed92fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.065020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983212c-01aa-40c3-afd5-dc4c3ed92fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.065302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983212c-01aa-40c3-afd5-dc4c3ed92fd3 admin admin] PATH_INFO: `/v3/projects/bef5c14f9ac349f08bb133ea1c1a5d09/users/60ec161a41b44e7fa14edc4593471911/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.076545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:13.082275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983212c-01aa-40c3-afd5-dc4c3ed92fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef5c14f9ac349f08bb133ea1c1a5d09, user_id=60ec161a41b44e7fa14edc4593471911, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.084021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983212c-01aa-40c3-afd5-dc4c3ed92fd3 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.230750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1043/2103] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:13 2022] PUT /identity/v3/projects/bef5c14f9ac349f08bb133ea1c1a5d09/users/60ec161a41b44e7fa14edc4593471911/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:13.234591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983212c-01aa-40c3-afd5-dc4c3ed92fd3 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.260044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8439c8343b754ae98ac0788b05cfe0e2'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.279495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8439c8343b754ae98ac0788b05cfe0e2'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.306301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8439c8343b754ae98ac0788b05cfe0e2'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.309021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8439c8343b754ae98ac0788b05cfe0e2'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.310708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8439c8343b754ae98ac0788b05cfe0e2'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.317231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983212c-01aa-40c3-afd5-dc4c3ed92fd3 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.322784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.323518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.324295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.324295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.327343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.327343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.337729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1044/2104] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:13.341777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.343518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:13.356676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c7038-e780-44ab-8ee4-7db90f8b9a1c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.361484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.362340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.362630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.362913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] PATH_INFO: `/v3/users/64d0e3be59b44781aa982d81511305a3` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.364187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d0e3be59b44781aa982d81511305a3)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.366133 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.422510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1061/2105] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:13 2022] DELETE /identity/v3/users/8439c8343b754ae98ac0788b05cfe0e2 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:13.423532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.429261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:13.493417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0837b-08f8-436c-980f-df24dba9a2c8 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.497974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14d5e1-c7b0-45f1-b1ef-abe4269dd702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.498862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d5e1-c7b0-45f1-b1ef-abe4269dd702 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.499151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d5e1-c7b0-45f1-b1ef-abe4269dd702 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.499434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d5e1-c7b0-45f1-b1ef-abe4269dd702 admin admin] PATH_INFO: `/v3/projects/bef5c14f9ac349f08bb133ea1c1a5d09/users/60ec161a41b44e7fa14edc4593471911/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.501474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14d5e1-c7b0-45f1-b1ef-abe4269dd702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef5c14f9ac349f08bb133ea1c1a5d09, user_id=60ec161a41b44e7fa14edc4593471911, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.504145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14d5e1-c7b0-45f1-b1ef-abe4269dd702 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.515681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1062/2106] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:13 2022] PUT /identity/v3/projects/bef5c14f9ac349f08bb133ea1c1a5d09/users/60ec161a41b44e7fa14edc4593471911/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:13.517176 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93791459-c81e-405a-a03f-ef644c03eb76 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.517521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93791459-c81e-405a-a03f-ef644c03eb76 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.517857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93791459-c81e-405a-a03f-ef644c03eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.538245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-93791459-c81e-405a-a03f-ef644c03eb76 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:13.539673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-93791459-c81e-405a-a03f-ef644c03eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:13.566591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d0e3be59b44781aa982d81511305a3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.595566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d0e3be59b44781aa982d81511305a3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.595566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d0e3be59b44781aa982d81511305a3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.595566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d0e3be59b44781aa982d81511305a3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.613824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1063/2107] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:13.615689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf4ee6-3312-402b-94ac-e7bff8f8c571 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.615689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf4ee6-3312-402b-94ac-e7bff8f8c571 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.622062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf4ee6-3312-402b-94ac-e7bff8f8c571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.654470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7edf4ee6-3312-402b-94ac-e7bff8f8c571 None None] MFA Rules not processed for user `60ec161a41b44e7fa14edc4593471911`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:13.655925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7edf4ee6-3312-402b-94ac-e7bff8f8c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:13.673905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d0e3be59b44781aa982d81511305a3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:13.694860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1045/2108] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:13 2022] DELETE /identity/v3/users/64d0e3be59b44781aa982d81511305a3 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:13.696034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.697482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:13.732412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:13.738991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1064/2109] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:13 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:13.738991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f3501-898f-43f5-971a-19651242cba2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.738991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f3501-898f-43f5-971a-19651242cba2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.738991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f3501-898f-43f5-971a-19651242cba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.760112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-192f3501-898f-43f5-971a-19651242cba2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:13.761794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-192f3501-898f-43f5-971a-19651242cba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:13.768037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b254f3f-be27-479f-8807-b26cae44edaa admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.775860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ab0bc-c552-4eda-b7d3-ce3ee18fb52a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.776517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ab0bc-c552-4eda-b7d3-ce3ee18fb52a service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.776890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ab0bc-c552-4eda-b7d3-ce3ee18fb52a service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.777170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ab0bc-c552-4eda-b7d3-ce3ee18fb52a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.822090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1065/2110] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:13.822946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f3501-898f-43f5-971a-19651242cba2 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.824408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-192f3501-898f-43f5-971a-19651242cba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:13.845552 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ab0bc-c552-4eda-b7d3-ce3ee18fb52a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.847122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ab0bc-c552-4eda-b7d3-ce3ee18fb52a service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.861465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1046/2111] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:13.862424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ab0bc-c552-4eda-b7d3-ce3ee18fb52a service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:13.873476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ab0bc-c552-4eda-b7d3-ce3ee18fb52a service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.878681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa02e9b-a6df-4f5d-8662-dc969ae436ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.880055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa02e9b-a6df-4f5d-8662-dc969ae436ba service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.880400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa02e9b-a6df-4f5d-8662-dc969ae436ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.880721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa02e9b-a6df-4f5d-8662-dc969ae436ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.891180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:13.893366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa02e9b-a6df-4f5d-8662-dc969ae436ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.894540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f3501-898f-43f5-971a-19651242cba2 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:13.894800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa02e9b-a6df-4f5d-8662-dc969ae436ba service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:13.900083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:13.901601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:13.902044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:13.902474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:13.910961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1047/2112] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:13.993670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:13.995461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:14.032183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1066/2113] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:13 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:14.344830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa02e9b-a6df-4f5d-8662-dc969ae436ba service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:14.354210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa02e9b-a6df-4f5d-8662-dc969ae436ba service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:14.359326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:14.359828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:14.360118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:14.360567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:14.362653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:14.420439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:14.422264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:14.430981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:14.433653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:14.435927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbf83a-fc10-4b22-8738-810bd54db9b3 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:14.441232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:14.442579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:14.442924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:14.443203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] PATH_INFO: `/v3/projects/609e77dfef414038b32f01bb25d177ee` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:14.444438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609e77dfef414038b32f01bb25d177ee)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:14.444992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1048/2114] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:14.446556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:14.516555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:14.518377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:14.533362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976e695-3b03-4770-b692-10a784e8d19b service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:14.538592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:14.542246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:14.542246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:14.542246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] PATH_INFO: `/v3/projects/5928f0aa1df044f78ace512911a04936` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:14.542246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5928f0aa1df044f78ace512911a04936)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:14.543401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:14.589679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609e77dfef414038b32f01bb25d177ee', 'request_id': 'req-5392a1bb-ec58-4491-bb34-a606d6daaff8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.691514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5928f0aa1df044f78ace512911a04936', 'request_id': 'req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.714112 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609e77dfef414038b32f01bb25d177ee', 'request_id': 'req-5392a1bb-ec58-4491-bb34-a606d6daaff8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.720211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609e77dfef414038b32f01bb25d177ee', 'request_id': 'req-5392a1bb-ec58-4491-bb34-a606d6daaff8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.732046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] The token cache is being invalidate because project 609e77dfef414038b32f01bb25d177ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:14.732340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.733014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.734586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1067/2115] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:14 2022] DELETE /identity/v3/projects/609e77dfef414038b32f01bb25d177ee => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:14.739465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:14.740906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:14.804013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392a1bb-ec58-4491-bb34-a606d6daaff8 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:14.812259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5928f0aa1df044f78ace512911a04936', 'request_id': 'req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.822949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5928f0aa1df044f78ace512911a04936', 'request_id': 'req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.838943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] The token cache is being invalidate because project 5928f0aa1df044f78ace512911a04936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:14.839283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.840009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:14.843828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1049/2116] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:14 2022] DELETE /identity/v3/projects/5928f0aa1df044f78ace512911a04936 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:14.846310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:14.848007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:14.877167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:14.878052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:14.878341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:14.879111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] PATH_INFO: `/v3/users/fdf36f0fb3004f73ba18018e2dbc6e94` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:14.880490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf36f0fb3004f73ba18018e2dbc6e94)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:14.882447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:14.956843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a4fb1-6bdf-450f-bce6-a305f7d9e93d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:14.966832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:14.967722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:14.968019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:14.968292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] PATH_INFO: `/v3/users/b030e73fa23f420ab34418c6583cd4ef` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:14.970660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b030e73fa23f420ab34418c6583cd4ef)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:14.972663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:15.054468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf36f0fb3004f73ba18018e2dbc6e94'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.067997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf36f0fb3004f73ba18018e2dbc6e94'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.075711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf36f0fb3004f73ba18018e2dbc6e94'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.076558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf36f0fb3004f73ba18018e2dbc6e94'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.077237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf36f0fb3004f73ba18018e2dbc6e94'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.149357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b030e73fa23f420ab34418c6583cd4ef'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.157011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b030e73fa23f420ab34418c6583cd4ef'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.157877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b030e73fa23f420ab34418c6583cd4ef'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.159062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b030e73fa23f420ab34418c6583cd4ef'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.183666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1068/2117] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:14 2022] DELETE /identity/v3/users/fdf36f0fb3004f73ba18018e2dbc6e94 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:15.245442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b030e73fa23f420ab34418c6583cd4ef'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:15.270649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1050/2118] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:14 2022] DELETE /identity/v3/users/b030e73fa23f420ab34418c6583cd4ef => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:15.367524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:15.768557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:15.770708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:15.804669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:15.807263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:15.856727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9f7e-6fea-4f21-88df-6a482adcb96b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:15.895743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3397-89bd-4c97-9b75-98a99ae5acce admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:15.900714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:15.901579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:15.901864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:15.902193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] PATH_INFO: `/v3/projects/3096c6350bc84a50bdc0cccce17aa75c` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:15.903381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3096c6350bc84a50bdc0cccce17aa75c)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:15.905361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:15.913912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:15.914886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:15.915254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:15.915534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] PATH_INFO: `/v3/projects/6c165280101947d8a0190f790c316324` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:15.916787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c165280101947d8a0190f790c316324)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:15.918726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.052564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3096c6350bc84a50bdc0cccce17aa75c', 'request_id': 'req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.058227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c165280101947d8a0190f790c316324', 'request_id': 'req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.146268 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3096c6350bc84a50bdc0cccce17aa75c', 'request_id': 'req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.154977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3096c6350bc84a50bdc0cccce17aa75c', 'request_id': 'req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.173965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] The token cache is being invalidate because project 3096c6350bc84a50bdc0cccce17aa75c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:16.174260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.174959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbb05942-cd4b-42b1-a4ca-5c3720d645fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.176820 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1051/2119] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:15 2022] DELETE /identity/v3/projects/3096c6350bc84a50bdc0cccce17aa75c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:16.185009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b025a32-1da5-443a-9d3f-7ddc2eeaaeb9 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.185381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b025a32-1da5-443a-9d3f-7ddc2eeaaeb9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.185666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b025a32-1da5-443a-9d3f-7ddc2eeaaeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.205097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7b025a32-1da5-443a-9d3f-7ddc2eeaaeb9 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:16.206307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7b025a32-1da5-443a-9d3f-7ddc2eeaaeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:16.264945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1052/2120] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:16.268067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c165280101947d8a0190f790c316324', 'request_id': 'req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.277259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcea344-b93f-4b8b-9c20-55178bb4269a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.277259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcea344-b93f-4b8b-9c20-55178bb4269a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.277259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcea344-b93f-4b8b-9c20-55178bb4269a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.277532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c165280101947d8a0190f790c316324', 'request_id': 'req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.287426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] The token cache is being invalidate because project 6c165280101947d8a0190f790c316324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:16.287696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.288371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dfb0d48e-5ec1-48e5-8199-41c9bea816e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:16.290061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1069/2121] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:15 2022] DELETE /identity/v3/projects/6c165280101947d8a0190f790c316324 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:16.294990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-cbcea344-b93f-4b8b-9c20-55178bb4269a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:16.300415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf5328-132c-44e2-aec2-9074a3ccd6ca None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.300498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cbcea344-b93f-4b8b-9c20-55178bb4269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:16.300604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf5328-132c-44e2-aec2-9074a3ccd6ca None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.300604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf5328-132c-44e2-aec2-9074a3ccd6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.307128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1053/2122] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:16.311009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcea344-b93f-4b8b-9c20-55178bb4269a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.318716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-58bf5328-132c-44e2-aec2-9074a3ccd6ca None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:16.319976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-58bf5328-132c-44e2-aec2-9074a3ccd6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:16.321496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcea344-b93f-4b8b-9c20-55178bb4269a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.329024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.329024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.329024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.329024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.329453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:16.329553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1070/2123] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:16.329642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.330499 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.336730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a56e3-0eed-4dfa-a061-5a143fadfb65 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.337185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a56e3-0eed-4dfa-a061-5a143fadfb65 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.337478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a56e3-0eed-4dfa-a061-5a143fadfb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.343501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1054/2124] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:36:16.347924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.357845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-ef6a56e3-0eed-4dfa-a061-5a143fadfb65 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:16.358470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ffdf6-90ad-4494-9703-6bf62fe26ba5 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.359089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ef6a56e3-0eed-4dfa-a061-5a143fadfb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:16.363275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd21b62-7751-4d32-a149-1448aea32d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.363949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd21b62-7751-4d32-a149-1448aea32d55 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.364235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd21b62-7751-4d32-a149-1448aea32d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.364513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd21b62-7751-4d32-a149-1448aea32d55 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.365224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd21b62-7751-4d32-a149-1448aea32d55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.367001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1071/2125] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:16.367113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd21b62-7751-4d32-a149-1448aea32d55 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.371229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a56e3-0eed-4dfa-a061-5a143fadfb65 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.380131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a56e3-0eed-4dfa-a061-5a143fadfb65 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.384907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.385573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.385855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.386135 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.388396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:16.388396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.388741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.402293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1072/2126] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:36:16.406677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.426881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b52d3-5455-4900-98f5-c91fea7a82d2 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.431736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0475-7dfa-43ad-ae57-bee66905580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.432601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0475-7dfa-43ad-ae57-bee66905580e admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.432911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0475-7dfa-43ad-ae57-bee66905580e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.433152 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0475-7dfa-43ad-ae57-bee66905580e admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.433843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e0475-7dfa-43ad-ae57-bee66905580e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.435712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e0475-7dfa-43ad-ae57-bee66905580e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.490388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1055/2127] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:16.495137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd21b62-7751-4d32-a149-1448aea32d55 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.508886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:16.558935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd21b62-7751-4d32-a149-1448aea32d55 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.563683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f54485-a21f-4e40-80c5-6b000f2fc3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.564652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54485-a21f-4e40-80c5-6b000f2fc3cc admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.564943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54485-a21f-4e40-80c5-6b000f2fc3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.565224 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54485-a21f-4e40-80c5-6b000f2fc3cc admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.565932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f54485-a21f-4e40-80c5-6b000f2fc3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.568345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f54485-a21f-4e40-80c5-6b000f2fc3cc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.571227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1073/2128] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:16.575781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0475-7dfa-43ad-ae57-bee66905580e admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.647404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0475-7dfa-43ad-ae57-bee66905580e admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.652273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86f32a-9033-4113-9c99-3e487442640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.653010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86f32a-9033-4113-9c99-3e487442640c admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.653349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86f32a-9033-4113-9c99-3e487442640c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.653637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86f32a-9033-4113-9c99-3e487442640c admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.654348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86f32a-9033-4113-9c99-3e487442640c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.656358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86f32a-9033-4113-9c99-3e487442640c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.674029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1056/2129] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:16.681688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f54485-a21f-4e40-80c5-6b000f2fc3cc admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.694094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f54485-a21f-4e40-80c5-6b000f2fc3cc admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.699555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549179-5bae-4b7c-b125-04a66676d049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.700235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549179-5bae-4b7c-b125-04a66676d049 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.701184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549179-5bae-4b7c-b125-04a66676d049 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.701487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549179-5bae-4b7c-b125-04a66676d049 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.702249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4549179-5bae-4b7c-b125-04a66676d049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.703840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4549179-5bae-4b7c-b125-04a66676d049 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.715354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1057/2130] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:16.722430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549179-5bae-4b7c-b125-04a66676d049 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.732745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549179-5bae-4b7c-b125-04a66676d049 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.737449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ce678-1853-479c-a9a2-f738634c1c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.738070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ce678-1853-479c-a9a2-f738634c1c27 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.738355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ce678-1853-479c-a9a2-f738634c1c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.738632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ce678-1853-479c-a9a2-f738634c1c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.739396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ce678-1853-479c-a9a2-f738634c1c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.741037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ce678-1853-479c-a9a2-f738634c1c27 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.751388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1058/2131] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:16.755849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ce678-1853-479c-a9a2-f738634c1c27 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.765042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ce678-1853-479c-a9a2-f738634c1c27 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.771259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa37c6-096f-4c11-acbe-fc30db64affd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.780435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa37c6-096f-4c11-acbe-fc30db64affd admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.780435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa37c6-096f-4c11-acbe-fc30db64affd admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.780435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa37c6-096f-4c11-acbe-fc30db64affd admin admin] PATH_INFO: `/v3/projects/7352fad617374a47aaec073756d3b128/users/4225af7f075d4f589c61b0b9f35d1658/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.781149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1074/2132] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:16.783502 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86f32a-9033-4113-9c99-3e487442640c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.790819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa37c6-096f-4c11-acbe-fc30db64affd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7352fad617374a47aaec073756d3b128, user_id=4225af7f075d4f589c61b0b9f35d1658, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.793869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86f32a-9033-4113-9c99-3e487442640c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.794638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa37c6-096f-4c11-acbe-fc30db64affd admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.800453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1261a-3203-4aea-b822-7b94d6d3e8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.801617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1261a-3203-4aea-b822-7b94d6d3e8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.801926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1261a-3203-4aea-b822-7b94d6d3e8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.802190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1261a-3203-4aea-b822-7b94d6d3e8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.802922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1261a-3203-4aea-b822-7b94d6d3e8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.810198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1261a-3203-4aea-b822-7b94d6d3e8a1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.822674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1075/2133] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:16.826630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1261a-3203-4aea-b822-7b94d6d3e8a1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.837531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1261a-3203-4aea-b822-7b94d6d3e8a1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.842297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38b5fd-d128-4156-b5ab-ddca1963a6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.842985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38b5fd-d128-4156-b5ab-ddca1963a6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.843313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38b5fd-d128-4156-b5ab-ddca1963a6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.843591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38b5fd-d128-4156-b5ab-ddca1963a6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.844315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38b5fd-d128-4156-b5ab-ddca1963a6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.845958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38b5fd-d128-4156-b5ab-ddca1963a6a3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.856739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1076/2134] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:16.860769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38b5fd-d128-4156-b5ab-ddca1963a6a3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:16.874609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38b5fd-d128-4156-b5ab-ddca1963a6a3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:16.878219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de193084-4f70-47e3-ac1c-8f8d8137aab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:16.879111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193084-4f70-47e3-ac1c-8f8d8137aab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.879396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193084-4f70-47e3-ac1c-8f8d8137aab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.880245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193084-4f70-47e3-ac1c-8f8d8137aab4 admin admin] PATH_INFO: `/v3/projects/df49ddfe878c42a5aa0d01d00dbca834/users/754e6c649fd948e180d5681d23891a25/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.895167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de193084-4f70-47e3-ac1c-8f8d8137aab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df49ddfe878c42a5aa0d01d00dbca834, user_id=754e6c649fd948e180d5681d23891a25, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:16.897921 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de193084-4f70-47e3-ac1c-8f8d8137aab4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:16.899871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1059/2135] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:16 2022] PUT /identity/v3/projects/7352fad617374a47aaec073756d3b128/users/4225af7f075d4f589c61b0b9f35d1658/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:16.912131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba26536-ac73-488b-835e-db72abdc1943 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:16.912422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba26536-ac73-488b-835e-db72abdc1943 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:16.912736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba26536-ac73-488b-835e-db72abdc1943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:16.939150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-eba26536-ac73-488b-835e-db72abdc1943 None None] MFA Rules not processed for user `4225af7f075d4f589c61b0b9f35d1658`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:16.940644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-eba26536-ac73-488b-835e-db72abdc1943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:16.990195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1077/2136] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:16 2022] PUT /identity/v3/projects/df49ddfe878c42a5aa0d01d00dbca834/users/754e6c649fd948e180d5681d23891a25/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:17.004422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1060/2137] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:17.005982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e876-a5c2-42cf-aebc-4c820d0dda68 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:17.006284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e876-a5c2-42cf-aebc-4c820d0dda68 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:17.006579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e876-a5c2-42cf-aebc-4c820d0dda68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:17.015396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba26536-ac73-488b-835e-db72abdc1943 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:17.017811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-eba26536-ac73-488b-835e-db72abdc1943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:17.036951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7cc3e876-a5c2-42cf-aebc-4c820d0dda68 None None] MFA Rules not processed for user `754e6c649fd948e180d5681d23891a25`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:17.038208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7cc3e876-a5c2-42cf-aebc-4c820d0dda68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:17.078550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba26536-ac73-488b-835e-db72abdc1943 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:17.108445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1078/2138] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:17.119512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3e876-a5c2-42cf-aebc-4c820d0dda68 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:17.120916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7cc3e876-a5c2-42cf-aebc-4c820d0dda68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:17.130916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366b2a6-4150-47ab-9e5f-dcd15f73efaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:17.134550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b2a6-4150-47ab-9e5f-dcd15f73efaa service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:17.134550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b2a6-4150-47ab-9e5f-dcd15f73efaa service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:17.134550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b2a6-4150-47ab-9e5f-dcd15f73efaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:17.179694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3e876-a5c2-42cf-aebc-4c820d0dda68 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:17.183511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e796e93-cdad-472b-ad0b-9a989f84d9df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:17.184199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e796e93-cdad-472b-ad0b-9a989f84d9df service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:17.184504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e796e93-cdad-472b-ad0b-9a989f84d9df service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:17.184790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e796e93-cdad-472b-ad0b-9a989f84d9df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:17.192786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366b2a6-4150-47ab-9e5f-dcd15f73efaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:17.194068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e796e93-cdad-472b-ad0b-9a989f84d9df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:17.194452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366b2a6-4150-47ab-9e5f-dcd15f73efaa service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:17.195511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e796e93-cdad-472b-ad0b-9a989f84d9df service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:17.204104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1061/2139] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:36:17.206470 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1079/2140] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:36:17.233325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366b2a6-4150-47ab-9e5f-dcd15f73efaa service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:17.302536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366b2a6-4150-47ab-9e5f-dcd15f73efaa service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:17.308389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:17.309298 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:17.309621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:17.309976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] PATH_INFO: `/v3/users/4225af7f075d4f589c61b0b9f35d1658` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:17.311183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4225af7f075d4f589c61b0b9f35d1658)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:17.313144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:17.624765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:17.745288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4225af7f075d4f589c61b0b9f35d1658'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:17.752182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4225af7f075d4f589c61b0b9f35d1658'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:17.752896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4225af7f075d4f589c61b0b9f35d1658'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:17.753508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4225af7f075d4f589c61b0b9f35d1658'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:17.908207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4225af7f075d4f589c61b0b9f35d1658'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:17.927337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1062/2141] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:17 2022] DELETE /identity/v3/users/4225af7f075d4f589c61b0b9f35d1658 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:17.931904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514517d-ad8c-4059-88dd-f569481dd9e7 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:17.932201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514517d-ad8c-4059-88dd-f569481dd9e7 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:17.932486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514517d-ad8c-4059-88dd-f569481dd9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:17.950421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f514517d-ad8c-4059-88dd-f569481dd9e7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:17.951604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f514517d-ad8c-4059-88dd-f569481dd9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:18.022448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1080/2142] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:18.032212 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:18.033712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:18.135289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75d0c-81fb-45a5-b42a-a7c3f3231d8d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:18.139664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100abb0a-0906-44ba-826d-fb8944f6af11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:18.142322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100abb0a-0906-44ba-826d-fb8944f6af11 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:18.142322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100abb0a-0906-44ba-826d-fb8944f6af11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:18.142322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100abb0a-0906-44ba-826d-fb8944f6af11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:18.151434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100abb0a-0906-44ba-826d-fb8944f6af11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:18.152914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100abb0a-0906-44ba-826d-fb8944f6af11 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:18.163601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1063/2143] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:18.217365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:18.370142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514517d-ad8c-4059-88dd-f569481dd9e7 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:18.371690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f514517d-ad8c-4059-88dd-f569481dd9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:18.384345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514517d-ad8c-4059-88dd-f569481dd9e7 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:18.389168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:18.390193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:18.390477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:18.390755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] PATH_INFO: `/v3/users/0d96ae2c0a894472a17fd24dc394a385` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:18.392116 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d96ae2c0a894472a17fd24dc394a385)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:18.394100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:18.623148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d96ae2c0a894472a17fd24dc394a385'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:18.640714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d96ae2c0a894472a17fd24dc394a385'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:18.648203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d96ae2c0a894472a17fd24dc394a385'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:18.649105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d96ae2c0a894472a17fd24dc394a385'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:18.649778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d96ae2c0a894472a17fd24dc394a385'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:18.752671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1081/2144] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:18 2022] DELETE /identity/v3/users/0d96ae2c0a894472a17fd24dc394a385 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:18.753349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eca12-9ad7-4848-860e-cf1ffb68d2e0 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:18.754343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eca12-9ad7-4848-860e-cf1ffb68d2e0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:18.755367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eca12-9ad7-4848-860e-cf1ffb68d2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:18.775826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-710eca12-9ad7-4848-860e-cf1ffb68d2e0 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:18.777139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-710eca12-9ad7-4848-860e-cf1ffb68d2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:18.784047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:18.786179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:18.872667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1064/2145] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:18.875896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4ab8d-a1ce-45ce-bd50-5eb0097899fc admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:18.880999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:18.882541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710eca12-9ad7-4848-860e-cf1ffb68d2e0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:18.882811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:18.883120 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:18.883406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] PATH_INFO: `/v3/projects/7352fad617374a47aaec073756d3b128` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:18.884624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-710eca12-9ad7-4848-860e-cf1ffb68d2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:18.884738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7352fad617374a47aaec073756d3b128)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:18.886778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:18.943179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710eca12-9ad7-4848-860e-cf1ffb68d2e0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:18.946839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef738499-1724-4d6f-beac-a535e6038b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:18.947490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738499-1724-4d6f-beac-a535e6038b48 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:18.947775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738499-1724-4d6f-beac-a535e6038b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:18.948093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738499-1724-4d6f-beac-a535e6038b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:18.957138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef738499-1724-4d6f-beac-a535e6038b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:18.958606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef738499-1724-4d6f-beac-a535e6038b48 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:18.967970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1065/2146] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:36:18.984811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:19.006877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7352fad617374a47aaec073756d3b128', 'request_id': 'req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.137206 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7352fad617374a47aaec073756d3b128', 'request_id': 'req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.145811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7352fad617374a47aaec073756d3b128', 'request_id': 'req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.159145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] The token cache is being invalidate because project 7352fad617374a47aaec073756d3b128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:19.159593 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.160333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4e26cf56-c6af-42ae-82a0-5e04ee5d6e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.162700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1082/2147] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:18 2022] DELETE /identity/v3/projects/7352fad617374a47aaec073756d3b128 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:19.167753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475d4a-680f-4d18-bcb1-31c7c8d7db05 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.168083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475d4a-680f-4d18-bcb1-31c7c8d7db05 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.168475 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475d4a-680f-4d18-bcb1-31c7c8d7db05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.197429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7c475d4a-680f-4d18-bcb1-31c7c8d7db05 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:19.200124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7c475d4a-680f-4d18-bcb1-31c7c8d7db05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:19.309555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:19.336567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1066/2148] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:19.344072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ee7a-cc30-48df-af23-b249a6a6e15a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.344442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ee7a-cc30-48df-af23-b249a6a6e15a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.344891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ee7a-cc30-48df-af23-b249a6a6e15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.364595 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-bc20ee7a-cc30-48df-af23-b249a6a6e15a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:19.366539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-bc20ee7a-cc30-48df-af23-b249a6a6e15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:19.374281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1083/2149] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:19.378861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c475d4a-680f-4d18-bcb1-31c7c8d7db05 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:19.394608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c475d4a-680f-4d18-bcb1-31c7c8d7db05 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:19.401969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:19.401969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.401969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.401969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.402657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:19.402916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:19.404711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:19.417239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1067/2150] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:36:19.421594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ee7a-cc30-48df-af23-b249a6a6e15a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:19.430944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ee7a-cc30-48df-af23-b249a6a6e15a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:19.435895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5fef8-3958-46d7-97b9-1c2a0c5c3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:19.436574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5fef8-3958-46d7-97b9-1c2a0c5c3eac admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.436859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5fef8-3958-46d7-97b9-1c2a0c5c3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.437137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5fef8-3958-46d7-97b9-1c2a0c5c3eac admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.437917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5fef8-3958-46d7-97b9-1c2a0c5c3eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:19.440070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5fef8-3958-46d7-97b9-1c2a0c5c3eac admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:19.505550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:19.506765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:19.519832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc296b0b-4865-4669-bfe0-6ace51ece022 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:19.525760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:19.526757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.527047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.527318 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] PATH_INFO: `/v3/projects/84b1f407cf0e45938d092cb59207f3e3` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.528781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b1f407cf0e45938d092cb59207f3e3)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:19.534039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:19.602065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1084/2151] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:19 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:19.606028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5fef8-3958-46d7-97b9-1c2a0c5c3eac admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:19.651038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b1f407cf0e45938d092cb59207f3e3', 'request_id': 'req-48a1034c-5f8a-4655-ae81-03859dd212b0'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.668597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5fef8-3958-46d7-97b9-1c2a0c5c3eac admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:19.722683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:19.723891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.724175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.724453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.725205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:19.727109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:19.776811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b1f407cf0e45938d092cb59207f3e3', 'request_id': 'req-48a1034c-5f8a-4655-ae81-03859dd212b0'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.786803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b1f407cf0e45938d092cb59207f3e3', 'request_id': 'req-48a1034c-5f8a-4655-ae81-03859dd212b0'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.800136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] The token cache is being invalidate because project 84b1f407cf0e45938d092cb59207f3e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:19.800437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.801171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:19.806613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1068/2152] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:19 2022] DELETE /identity/v3/projects/84b1f407cf0e45938d092cb59207f3e3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:19.807352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:19.809540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:19.814929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:19.823678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1034c-5f8a-4655-ae81-03859dd212b0 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:19.828469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:19.829384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.829692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.829971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] PATH_INFO: `/v3/users/60ec161a41b44e7fa14edc4593471911` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.831159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ec161a41b44e7fa14edc4593471911)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:19.833128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:19.841010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1085/2153] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:19 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:19.851006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:19.852684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:19.864887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc298f-c975-4f4d-9f17-67ea2921827a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:19.869527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153e9c6-8765-4ee9-bfb0-e759e35b5931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:19.870180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153e9c6-8765-4ee9-bfb0-e759e35b5931 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.870469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153e9c6-8765-4ee9-bfb0-e759e35b5931 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.870756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153e9c6-8765-4ee9-bfb0-e759e35b5931 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.871484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153e9c6-8765-4ee9-bfb0-e759e35b5931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:19.873106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153e9c6-8765-4ee9-bfb0-e759e35b5931 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:19.882918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1086/2154] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:19.886684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153e9c6-8765-4ee9-bfb0-e759e35b5931 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:19.897338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153e9c6-8765-4ee9-bfb0-e759e35b5931 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:19.902109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d1191-e5ca-4ade-a300-cfeac9fc5151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:19.902700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d1191-e5ca-4ade-a300-cfeac9fc5151 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.902983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d1191-e5ca-4ade-a300-cfeac9fc5151 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.903266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d1191-e5ca-4ade-a300-cfeac9fc5151 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.903988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d1191-e5ca-4ade-a300-cfeac9fc5151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:19.905773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d1191-e5ca-4ade-a300-cfeac9fc5151 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:19.916610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1087/2155] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:19.920869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d1191-e5ca-4ade-a300-cfeac9fc5151 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:19.934274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d1191-e5ca-4ade-a300-cfeac9fc5151 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:19.937378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592d019-f32e-4789-9101-f72025317a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:19.938307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592d019-f32e-4789-9101-f72025317a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:19.938591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592d019-f32e-4789-9101-f72025317a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:19.938865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592d019-f32e-4789-9101-f72025317a3f admin admin] PATH_INFO: `/v3/projects/6e6eaec274fd4b8295bf2a9fe68fbf26/users/108d5002338544cf8afedeac10749a82/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:19.953916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592d019-f32e-4789-9101-f72025317a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6eaec274fd4b8295bf2a9fe68fbf26, user_id=108d5002338544cf8afedeac10749a82, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:19.956602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592d019-f32e-4789-9101-f72025317a3f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:20.001515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ec161a41b44e7fa14edc4593471911'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.008244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ec161a41b44e7fa14edc4593471911'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.008964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ec161a41b44e7fa14edc4593471911'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.009565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ec161a41b44e7fa14edc4593471911'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.073514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1088/2156] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:19 2022] PUT /identity/v3/projects/6e6eaec274fd4b8295bf2a9fe68fbf26/users/108d5002338544cf8afedeac10749a82/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:20.084261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901550b1-03d2-4cfb-a2d3-997ef8e9cda5 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:20.084627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901550b1-03d2-4cfb-a2d3-997ef8e9cda5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:20.084912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901550b1-03d2-4cfb-a2d3-997ef8e9cda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:20.092632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ec161a41b44e7fa14edc4593471911'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.115729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-901550b1-03d2-4cfb-a2d3-997ef8e9cda5 None None] MFA Rules not processed for user `108d5002338544cf8afedeac10749a82`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:20.116742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-901550b1-03d2-4cfb-a2d3-997ef8e9cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:20.117505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1069/2157] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:19 2022] DELETE /identity/v3/users/60ec161a41b44e7fa14edc4593471911 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:20.192280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1089/2158] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:20.202920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:20.204424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:20.245450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:20.264088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e75113-afe2-4fea-acb8-f9cb8936557a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:20.267791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2f709-dcaa-4f36-a8cf-54d16c25325e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:20.268452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2f709-dcaa-4f36-a8cf-54d16c25325e service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:20.268764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2f709-dcaa-4f36-a8cf-54d16c25325e service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:20.277742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2f709-dcaa-4f36-a8cf-54d16c25325e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:20.278023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2f709-dcaa-4f36-a8cf-54d16c25325e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:20.279456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2f709-dcaa-4f36-a8cf-54d16c25325e service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:20.288283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1070/2159] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:20.495444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:20.659213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901550b1-03d2-4cfb-a2d3-997ef8e9cda5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:20.660747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-901550b1-03d2-4cfb-a2d3-997ef8e9cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:20.741356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901550b1-03d2-4cfb-a2d3-997ef8e9cda5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:20.746243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:20.747168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:20.747806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:20.748271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] PATH_INFO: `/v3/projects/bef5c14f9ac349f08bb133ea1c1a5d09` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:20.749858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef5c14f9ac349f08bb133ea1c1a5d09)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:20.751828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:20.870742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef5c14f9ac349f08bb133ea1c1a5d09', 'request_id': 'req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.971060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef5c14f9ac349f08bb133ea1c1a5d09', 'request_id': 'req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.971414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:20.977336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef5c14f9ac349f08bb133ea1c1a5d09', 'request_id': 'req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.988454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] The token cache is being invalidate because project bef5c14f9ac349f08bb133ea1c1a5d09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:20.989305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.989932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-9ab7b860-6514-4eff-9367-81c2aeeb6ab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:20.991456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1090/2160] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:20 2022] DELETE /identity/v3/projects/bef5c14f9ac349f08bb133ea1c1a5d09 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:20.996821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e633b-15ad-41e1-8e14-00fab3e36e4e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:20.997117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e633b-15ad-41e1-8e14-00fab3e36e4e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:20.997401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e633b-15ad-41e1-8e14-00fab3e36e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.015974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-603e633b-15ad-41e1-8e14-00fab3e36e4e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:21.017218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-603e633b-15ad-41e1-8e14-00fab3e36e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:21.075075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1071/2161] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:21.082650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb7696-5e24-4fc0-b1b2-6b8163e47a8f None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.082945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb7696-5e24-4fc0-b1b2-6b8163e47a8f None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.083368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb7696-5e24-4fc0-b1b2-6b8163e47a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.102432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-07eb7696-5e24-4fc0-b1b2-6b8163e47a8f None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:21.103634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-07eb7696-5e24-4fc0-b1b2-6b8163e47a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:21.111453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1091/2162] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:21.116375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e633b-15ad-41e1-8e14-00fab3e36e4e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:21.125971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e633b-15ad-41e1-8e14-00fab3e36e4e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:21.130729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:21.131525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.131818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.132102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.132873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:21.133191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:21.145484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:21.147634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1072/2163] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:36:21.151778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb7696-5e24-4fc0-b1b2-6b8163e47a8f None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:21.161612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb7696-5e24-4fc0-b1b2-6b8163e47a8f None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:21.167486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e304e-8fe3-429e-8b4b-2fd93914cf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:21.168564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e304e-8fe3-429e-8b4b-2fd93914cf51 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.169024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e304e-8fe3-429e-8b4b-2fd93914cf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.169308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e304e-8fe3-429e-8b4b-2fd93914cf51 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.170037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e304e-8fe3-429e-8b4b-2fd93914cf51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:21.171935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e304e-8fe3-429e-8b4b-2fd93914cf51 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:21.291873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1092/2164] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:21 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:21.293587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:21.304234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fd508-3e57-471e-9a3c-5a109dadab4e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:21.361161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af134-921f-44c0-b386-1bf0e5b4cb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:21.361836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af134-921f-44c0-b386-1bf0e5b4cb75 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.362127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af134-921f-44c0-b386-1bf0e5b4cb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.362409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af134-921f-44c0-b386-1bf0e5b4cb75 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.363113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af134-921f-44c0-b386-1bf0e5b4cb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:21.365642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af134-921f-44c0-b386-1bf0e5b4cb75 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:21.502844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1073/2165] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:21 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:21.510432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e304e-8fe3-429e-8b4b-2fd93914cf51 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:21.516757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e304e-8fe3-429e-8b4b-2fd93914cf51 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:21.521846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf634c-423b-45a4-a3bf-399fb59e3b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:21.522451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf634c-423b-45a4-a3bf-399fb59e3b8d admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.522740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf634c-423b-45a4-a3bf-399fb59e3b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.523019 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf634c-423b-45a4-a3bf-399fb59e3b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.524017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf634c-423b-45a4-a3bf-399fb59e3b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:21.525395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf634c-423b-45a4-a3bf-399fb59e3b8d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:21.536740 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1093/2166] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:21.540781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af134-921f-44c0-b386-1bf0e5b4cb75 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:21.551205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af134-921f-44c0-b386-1bf0e5b4cb75 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:21.556937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9a92-6e62-4507-918d-90457c04d0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:21.565008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9a92-6e62-4507-918d-90457c04d0de admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.565008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9a92-6e62-4507-918d-90457c04d0de admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.565008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9a92-6e62-4507-918d-90457c04d0de admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.565008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d9a92-6e62-4507-918d-90457c04d0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:21.565008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d9a92-6e62-4507-918d-90457c04d0de admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:21.571521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1074/2167] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:21.575713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf634c-423b-45a4-a3bf-399fb59e3b8d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:21.593978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf634c-423b-45a4-a3bf-399fb59e3b8d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:21.593978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8fa62-3909-4304-9bc7-4bf59ec6ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:21.593978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8fa62-3909-4304-9bc7-4bf59ec6ca32 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.593978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8fa62-3909-4304-9bc7-4bf59ec6ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.594323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8fa62-3909-4304-9bc7-4bf59ec6ca32 admin admin] PATH_INFO: `/v3/projects/f61e53971e024c6484791da614faa3a9/users/4031f95712fc4e478a92675fd0ba8fb4/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.614752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8fa62-3909-4304-9bc7-4bf59ec6ca32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61e53971e024c6484791da614faa3a9, user_id=4031f95712fc4e478a92675fd0ba8fb4, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:21.617494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8fa62-3909-4304-9bc7-4bf59ec6ca32 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:21.624285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:21.709292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1094/2168] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:21 2022] PUT /identity/v3/projects/f61e53971e024c6484791da614faa3a9/users/4031f95712fc4e478a92675fd0ba8fb4/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:21.720711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357940be-efb8-42dd-b18c-ccb659d12d90 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.722700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357940be-efb8-42dd-b18c-ccb659d12d90 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.722700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357940be-efb8-42dd-b18c-ccb659d12d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.748736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-357940be-efb8-42dd-b18c-ccb659d12d90 None None] MFA Rules not processed for user `4031f95712fc4e478a92675fd0ba8fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:21.754274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-357940be-efb8-42dd-b18c-ccb659d12d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:21.812568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1075/2169] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:21.817967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8fa62-3909-4304-9bc7-4bf59ec6ca32 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:21.883660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8fa62-3909-4304-9bc7-4bf59ec6ca32 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:21.888266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd62b7-d492-4efd-800f-0986385c0894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:21.889088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd62b7-d492-4efd-800f-0986385c0894 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:21.889311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd62b7-d492-4efd-800f-0986385c0894 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:21.889594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd62b7-d492-4efd-800f-0986385c0894 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:21.893997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd62b7-d492-4efd-800f-0986385c0894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:21.893997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd62b7-d492-4efd-800f-0986385c0894 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:21.995461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1095/2170] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:21 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:21.999590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357940be-efb8-42dd-b18c-ccb659d12d90 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:22.010881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357940be-efb8-42dd-b18c-ccb659d12d90 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:22.066553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea6d9e-47b4-4d3f-83c0-3c5761601536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:22.067267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea6d9e-47b4-4d3f-83c0-3c5761601536 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:22.068054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea6d9e-47b4-4d3f-83c0-3c5761601536 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:22.068333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea6d9e-47b4-4d3f-83c0-3c5761601536 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:22.069061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea6d9e-47b4-4d3f-83c0-3c5761601536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:22.071764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea6d9e-47b4-4d3f-83c0-3c5761601536 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:22.147750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1076/2171] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:21 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:22.151651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd62b7-d492-4efd-800f-0986385c0894 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:22.153596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:22.161850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd62b7-d492-4efd-800f-0986385c0894 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:22.166467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950bd2f-5bbb-4952-b2ba-b9bbedb96122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:22.167118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950bd2f-5bbb-4952-b2ba-b9bbedb96122 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:22.167401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950bd2f-5bbb-4952-b2ba-b9bbedb96122 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:22.167677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950bd2f-5bbb-4952-b2ba-b9bbedb96122 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:22.168396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950bd2f-5bbb-4952-b2ba-b9bbedb96122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:22.170053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950bd2f-5bbb-4952-b2ba-b9bbedb96122 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:22.180970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1096/2172] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:22.185107 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea6d9e-47b4-4d3f-83c0-3c5761601536 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:22.195779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea6d9e-47b4-4d3f-83c0-3c5761601536 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:22.200315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205262-49a8-4805-8a9f-82ab0974d3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:22.201211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205262-49a8-4805-8a9f-82ab0974d3a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:22.203040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205262-49a8-4805-8a9f-82ab0974d3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:22.203040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205262-49a8-4805-8a9f-82ab0974d3a0 admin admin] PATH_INFO: `/v3/projects/6aa17e027b6d45caab19e14cf99abd63/users/270269605fb245fd996b69937685df2d/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:22.216427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1205262-49a8-4805-8a9f-82ab0974d3a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa17e027b6d45caab19e14cf99abd63, user_id=270269605fb245fd996b69937685df2d, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:22.219128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1205262-49a8-4805-8a9f-82ab0974d3a0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:22.222298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:22.285191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1077/2173] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:22 2022] PUT /identity/v3/projects/6aa17e027b6d45caab19e14cf99abd63/users/270269605fb245fd996b69937685df2d/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:22.289263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950bd2f-5bbb-4952-b2ba-b9bbedb96122 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:22.299289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950bd2f-5bbb-4952-b2ba-b9bbedb96122 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:22.355712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c7ee6-0373-4386-a9a8-32d352e4a734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:22.356319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c7ee6-0373-4386-a9a8-32d352e4a734 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:22.356643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c7ee6-0373-4386-a9a8-32d352e4a734 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:22.356933 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c7ee6-0373-4386-a9a8-32d352e4a734 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:22.357696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c7ee6-0373-4386-a9a8-32d352e4a734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:22.359326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c7ee6-0373-4386-a9a8-32d352e4a734 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:22.369532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1097/2174] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:22.373805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205262-49a8-4805-8a9f-82ab0974d3a0 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:22.384706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205262-49a8-4805-8a9f-82ab0974d3a0 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:22.389282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:22.390158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:22.390444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:22.390722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] PATH_INFO: `/v3/projects/6aa17e027b6d45caab19e14cf99abd63/users/270269605fb245fd996b69937685df2d/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:22.392759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa17e027b6d45caab19e14cf99abd63, user_id=270269605fb245fd996b69937685df2d, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:22.395402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:22.406258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1078/2175] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:22 2022] PUT /identity/v3/projects/6aa17e027b6d45caab19e14cf99abd63/users/270269605fb245fd996b69937685df2d/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:22.417098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3936-fc70-4696-92eb-dc5ec20d2fbd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:22.421309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3936-fc70-4696-92eb-dc5ec20d2fbd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:22.421309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3936-fc70-4696-92eb-dc5ec20d2fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:22.444742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4fca3936-fc70-4696-92eb-dc5ec20d2fbd None None] MFA Rules not processed for user `270269605fb245fd996b69937685df2d`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:22.445959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4fca3936-fc70-4696-92eb-dc5ec20d2fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:22.499735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:22.517147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1098/2176] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:22.522461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:22.528541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:22.530234 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:22.588239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095e35f-7a66-4d23-8c42-746daa8c182a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:22.591930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:22.592653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:22.592942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:22.593222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:22.602897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:22.604374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:22.613887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1079/2177] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:22 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:22.721535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:22.868684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:23.216981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:26.534604 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113427c-def6-491a-89be-0208adbf0952 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:26.535245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113427c-def6-491a-89be-0208adbf0952 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:26.535527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113427c-def6-491a-89be-0208adbf0952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:26.556730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-3113427c-def6-491a-89be-0208adbf0952 None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:26.558342 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3113427c-def6-491a-89be-0208adbf0952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:26.627806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1099/2178] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:36:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:26.671673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:26.673188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:26.747294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa5a38-5c77-42cb-9b18-7077d0e820a0 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:26.752287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03d754-72ea-44df-a832-cef4cbecb45a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:26.752974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03d754-72ea-44df-a832-cef4cbecb45a service swift] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:26.753262 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03d754-72ea-44df-a832-cef4cbecb45a service swift] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:26.753546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03d754-72ea-44df-a832-cef4cbecb45a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:26.761997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03d754-72ea-44df-a832-cef4cbecb45a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:26.763580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03d754-72ea-44df-a832-cef4cbecb45a service swift] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:26.772385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1080/2179] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:36:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:36:26.949983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113427c-def6-491a-89be-0208adbf0952 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:27.008436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:27.013980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113427c-def6-491a-89be-0208adbf0952 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:27.019034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:27.019926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:27.020205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:27.021066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] PATH_INFO: `/v3/users/4031f95712fc4e478a92675fd0ba8fb4` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:27.022322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4031f95712fc4e478a92675fd0ba8fb4)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:27.024228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:27.224840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4031f95712fc4e478a92675fd0ba8fb4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:27.237972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4031f95712fc4e478a92675fd0ba8fb4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:27.247067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4031f95712fc4e478a92675fd0ba8fb4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:27.247067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4031f95712fc4e478a92675fd0ba8fb4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:27.247559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4031f95712fc4e478a92675fd0ba8fb4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:27.309520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1100/2180] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:26 2022] DELETE /identity/v3/users/4031f95712fc4e478a92675fd0ba8fb4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:27.315170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f75fe-3ab2-4273-827a-20935ff53cec None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:27.315170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f75fe-3ab2-4273-827a-20935ff53cec None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:27.315170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f75fe-3ab2-4273-827a-20935ff53cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:27.333922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-086f75fe-3ab2-4273-827a-20935ff53cec None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:27.335126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-086f75fe-3ab2-4273-827a-20935ff53cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:27.373829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:27.400671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1081/2181] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:27.410241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:27.411722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:27.477242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6429f27-b5a7-4852-b28c-a50a3336ed50 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:27.484157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:27.484844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:27.485130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:27.485541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:27.495923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:27.497501 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:27.508490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1101/2182] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:36:27.639733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5945-3a3f-4ba8-a805-b03504a6ae6e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:27.640073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5945-3a3f-4ba8-a805-b03504a6ae6e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:27.640359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5945-3a3f-4ba8-a805-b03504a6ae6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:27.658805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-5bfa5945-3a3f-4ba8-a805-b03504a6ae6e None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:27.660036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5bfa5945-3a3f-4ba8-a805-b03504a6ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:27.717526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1082/2183] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jun 16 00:36:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:27.730680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:27.733333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:27.807408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0f1ca-9855-474c-bc08-249aa47460fe service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:27.812517 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199bd4c-0d52-41e6-96fe-c05dc86cbcea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:27.813256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199bd4c-0d52-41e6-96fe-c05dc86cbcea service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:27.813587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199bd4c-0d52-41e6-96fe-c05dc86cbcea service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:27.813885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199bd4c-0d52-41e6-96fe-c05dc86cbcea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:27.823627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199bd4c-0d52-41e6-96fe-c05dc86cbcea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:27.825819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199bd4c-0d52-41e6-96fe-c05dc86cbcea service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:27.834626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1102/2184] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:36:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:36:27.980963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5945-3a3f-4ba8-a805-b03504a6ae6e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:27.982431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5bfa5945-3a3f-4ba8-a805-b03504a6ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:27.999615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5945-3a3f-4ba8-a805-b03504a6ae6e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:28.004691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:28.005585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:28.005873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:28.006145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] PATH_INFO: `/v3/projects/f61e53971e024c6484791da614faa3a9` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:28.007335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61e53971e024c6484791da614faa3a9)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:28.009270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:28.149366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61e53971e024c6484791da614faa3a9', 'request_id': 'req-7c0f7b9c-7de2-404f-b29d-876746289983'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.235245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61e53971e024c6484791da614faa3a9', 'request_id': 'req-7c0f7b9c-7de2-404f-b29d-876746289983'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.241305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61e53971e024c6484791da614faa3a9', 'request_id': 'req-7c0f7b9c-7de2-404f-b29d-876746289983'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.252352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] The token cache is being invalidate because project f61e53971e024c6484791da614faa3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:28.252703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.253313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7c0f7b9c-7de2-404f-b29d-876746289983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.254870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1083/2185] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:27 2022] DELETE /identity/v3/projects/f61e53971e024c6484791da614faa3a9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:28.258912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199bd4c-0d52-41e6-96fe-c05dc86cbcea service swift] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:28.269397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199bd4c-0d52-41e6-96fe-c05dc86cbcea service swift] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:28.274982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:28.333177 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:28.334004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:28.334294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:28.334575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] PATH_INFO: `/v3/users/270269605fb245fd996b69937685df2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:28.335764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] RBAC: Authorizing `identity:delete_user(user_id=270269605fb245fd996b69937685df2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:28.337730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:28.550798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967886a-dd50-40fa-a6ce-78f4936ab402 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:28.551089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967886a-dd50-40fa-a6ce-78f4936ab402 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:28.551372 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967886a-dd50-40fa-a6ce-78f4936ab402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:28.569662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9967886a-dd50-40fa-a6ce-78f4936ab402 None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:28.570851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9967886a-dd50-40fa-a6ce-78f4936ab402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:28.584886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '270269605fb245fd996b69937685df2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.603671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '270269605fb245fd996b69937685df2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.610707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '270269605fb245fd996b69937685df2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.611418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '270269605fb245fd996b69937685df2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.612020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '270269605fb245fd996b69937685df2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:28.635415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1084/2186] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:36:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:28.647184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967886a-dd50-40fa-a6ce-78f4936ab402 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:28.648923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9967886a-dd50-40fa-a6ce-78f4936ab402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:28.662495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:28.722542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967886a-dd50-40fa-a6ce-78f4936ab402 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:28.728743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb53fa-2717-4585-a209-ad036ab3e159 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:28.729480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb53fa-2717-4585-a209-ad036ab3e159 service swift] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:28.729767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb53fa-2717-4585-a209-ad036ab3e159 service swift] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:28.730053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb53fa-2717-4585-a209-ad036ab3e159 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:28.732971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1103/2187] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:28 2022] DELETE /identity/v3/users/270269605fb245fd996b69937685df2d => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:28.742723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb53fa-2717-4585-a209-ad036ab3e159 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:28.744979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb53fa-2717-4585-a209-ad036ab3e159 service swift] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:28.754873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1085/2188] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:36:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:36:29.223512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:29.225606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:29.309858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef22f6-2d18-4df7-a8d1-d0e7409a729d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:29.314963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:29.315831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:29.316174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:29.316464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] PATH_INFO: `/v3/projects/6aa17e027b6d45caab19e14cf99abd63` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:29.317749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aa17e027b6d45caab19e14cf99abd63)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:29.319803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:29.377564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f01c2-ea40-4d5a-bb25-fbc38877706d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:29.377862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f01c2-ea40-4d5a-bb25-fbc38877706d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:29.378172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f01c2-ea40-4d5a-bb25-fbc38877706d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:29.396143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7f3f01c2-ea40-4d5a-bb25-fbc38877706d None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:29.397342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7f3f01c2-ea40-4d5a-bb25-fbc38877706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:29.415936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:29.431947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aa17e027b6d45caab19e14cf99abd63', 'request_id': 'req-d0748e88-bddd-49da-9883-db076ad423a4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:29.455730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1086/2189] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jun 16 00:36:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:29.468075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f01c2-ea40-4d5a-bb25-fbc38877706d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:29.541428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f01c2-ea40-4d5a-bb25-fbc38877706d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:29.585181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aa17e027b6d45caab19e14cf99abd63', 'request_id': 'req-d0748e88-bddd-49da-9883-db076ad423a4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:29.587123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:29.591862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aa17e027b6d45caab19e14cf99abd63', 'request_id': 'req-d0748e88-bddd-49da-9883-db076ad423a4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:29.604111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b84043-d2e6-4bd8-b0d9-38f79d704770 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:29.614990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84043-d2e6-4bd8-b0d9-38f79d704770 service swift] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:29.614990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84043-d2e6-4bd8-b0d9-38f79d704770 service swift] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:29.614990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84043-d2e6-4bd8-b0d9-38f79d704770 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:29.620662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] The token cache is being invalidate because project 6aa17e027b6d45caab19e14cf99abd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:29.621197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:29.621885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d0748e88-bddd-49da-9883-db076ad423a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:29.623597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1104/2190] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:29 2022] DELETE /identity/v3/projects/6aa17e027b6d45caab19e14cf99abd63 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:29.628454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb8172-341d-4737-b4c5-5629a6e5d391 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:29.628733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb8172-341d-4737-b4c5-5629a6e5d391 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:29.629047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb8172-341d-4737-b4c5-5629a6e5d391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:29.649800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-19eb8172-341d-4737-b4c5-5629a6e5d391 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:29.650938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-19eb8172-341d-4737-b4c5-5629a6e5d391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:29.662199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b84043-d2e6-4bd8-b0d9-38f79d704770 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:29.663782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b84043-d2e6-4bd8-b0d9-38f79d704770 service swift] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:29.672403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1087/2191] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:36:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:36:29.717196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1105/2192] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:29.725619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfef19-2d8b-4789-a730-e6fc15473e9b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:29.726075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfef19-2d8b-4789-a730-e6fc15473e9b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:29.726397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfef19-2d8b-4789-a730-e6fc15473e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:29.745323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-85bfef19-2d8b-4789-a730-e6fc15473e9b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:29.746838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-85bfef19-2d8b-4789-a730-e6fc15473e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:29.755491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1088/2193] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:29.761262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb8172-341d-4737-b4c5-5629a6e5d391 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:29.772966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb8172-341d-4737-b4c5-5629a6e5d391 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:29.793207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:29.794095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:29.794483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:29.795050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:29.797072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:29.797301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:29.798953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:29.820298 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1106/2194] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:36:29.824462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfef19-2d8b-4789-a730-e6fc15473e9b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:29.834524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfef19-2d8b-4789-a730-e6fc15473e9b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:29.840945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7434d43-53ae-4ded-b3ef-2cc9baf458c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:29.841596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7434d43-53ae-4ded-b3ef-2cc9baf458c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:29.841884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7434d43-53ae-4ded-b3ef-2cc9baf458c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:29.842167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7434d43-53ae-4ded-b3ef-2cc9baf458c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:29.842863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7434d43-53ae-4ded-b3ef-2cc9baf458c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:29.844786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7434d43-53ae-4ded-b3ef-2cc9baf458c8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:29.934659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1089/2195] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:29 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:29.938597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:29.949352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2b2de-87a0-40d1-88f3-f5a5ccb81596 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.026528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42534-a131-41a1-95ed-fcc3285f5858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.027296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42534-a131-41a1-95ed-fcc3285f5858 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.027648 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42534-a131-41a1-95ed-fcc3285f5858 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.027996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42534-a131-41a1-95ed-fcc3285f5858 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.028805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b42534-a131-41a1-95ed-fcc3285f5858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.030846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b42534-a131-41a1-95ed-fcc3285f5858 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:30.130923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1107/2196] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:29 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:30.132172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7434d43-53ae-4ded-b3ef-2cc9baf458c8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:30.142768 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7434d43-53ae-4ded-b3ef-2cc9baf458c8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.149774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f552d3-0498-4299-9b15-6482904078ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.150715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f552d3-0498-4299-9b15-6482904078ad admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.151299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f552d3-0498-4299-9b15-6482904078ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.151658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f552d3-0498-4299-9b15-6482904078ad admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.152315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f552d3-0498-4299-9b15-6482904078ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.155643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f552d3-0498-4299-9b15-6482904078ad admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:30.168387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1090/2197] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:30.172800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42534-a131-41a1-95ed-fcc3285f5858 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:30.183772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42534-a131-41a1-95ed-fcc3285f5858 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.188742 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a936-779d-4b3a-86a5-e64328057a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.189723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a936-779d-4b3a-86a5-e64328057a8d admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.190157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a936-779d-4b3a-86a5-e64328057a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.190447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a936-779d-4b3a-86a5-e64328057a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.191295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a936-779d-4b3a-86a5-e64328057a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.193020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a936-779d-4b3a-86a5-e64328057a8d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:30.221458 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1108/2198] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:30.226314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f552d3-0498-4299-9b15-6482904078ad admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:30.236914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f552d3-0498-4299-9b15-6482904078ad admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.242781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522fef6-82cc-44a4-90f0-e6652ce32250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.243662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522fef6-82cc-44a4-90f0-e6652ce32250 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.243945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522fef6-82cc-44a4-90f0-e6652ce32250 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.244222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522fef6-82cc-44a4-90f0-e6652ce32250 admin admin] PATH_INFO: `/v3/projects/6062f9a0e12d4c4596ad9c1f243a72ca/users/b620f38c905a4240a665bf4de033396f/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.259041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522fef6-82cc-44a4-90f0-e6652ce32250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6062f9a0e12d4c4596ad9c1f243a72ca, user_id=b620f38c905a4240a665bf4de033396f, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.263070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522fef6-82cc-44a4-90f0-e6652ce32250 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:30.346579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1091/2199] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:30 2022] PUT /identity/v3/projects/6062f9a0e12d4c4596ad9c1f243a72ca/users/b620f38c905a4240a665bf4de033396f/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:30.358091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add0c35-93ea-4f57-a54f-cfcb36cde344 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.358391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add0c35-93ea-4f57-a54f-cfcb36cde344 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.358679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add0c35-93ea-4f57-a54f-cfcb36cde344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.388084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1add0c35-93ea-4f57-a54f-cfcb36cde344 None None] MFA Rules not processed for user `b620f38c905a4240a665bf4de033396f`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:30.389322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1add0c35-93ea-4f57-a54f-cfcb36cde344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:30.399611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:30.480241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1109/2200] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:30.491771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522fef6-82cc-44a4-90f0-e6652ce32250 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:30.579569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522fef6-82cc-44a4-90f0-e6652ce32250 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.587211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad189e-fa39-4143-9d9c-e88e78040bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.589757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad189e-fa39-4143-9d9c-e88e78040bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.589757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad189e-fa39-4143-9d9c-e88e78040bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.589757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad189e-fa39-4143-9d9c-e88e78040bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.589757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad189e-fa39-4143-9d9c-e88e78040bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.591160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad189e-fa39-4143-9d9c-e88e78040bb3 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:30.710563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1092/2201] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:30 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:30.714466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add0c35-93ea-4f57-a54f-cfcb36cde344 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:30.726305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add0c35-93ea-4f57-a54f-cfcb36cde344 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.783878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.784574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.784861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.785146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.785850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.787796 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:30.870441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1110/2202] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:30 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:30.875233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad189e-fa39-4143-9d9c-e88e78040bb3 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:30.891754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad189e-fa39-4143-9d9c-e88e78040bb3 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.898550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ec19d-3aaa-42c6-af23-d8272856bd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.900711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ec19d-3aaa-42c6-af23-d8272856bd8d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.901000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ec19d-3aaa-42c6-af23-d8272856bd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.901287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ec19d-3aaa-42c6-af23-d8272856bd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.901665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:30.902325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ec19d-3aaa-42c6-af23-d8272856bd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.903579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:30.903686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ec19d-3aaa-42c6-af23-d8272856bd8d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:30.909545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:30.916180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1093/2203] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:30.921265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fd596-6779-4c98-919f-a0115f00fb7c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.924304 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ec19d-3aaa-42c6-af23-d8272856bd8d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:30.927509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.928417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.928745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.929025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] PATH_INFO: `/v3/users/754e6c649fd948e180d5681d23891a25` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.930353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=754e6c649fd948e180d5681d23891a25)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.936391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:30.956086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ec19d-3aaa-42c6-af23-d8272856bd8d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:30.972286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8253b7-6df4-4064-ab0c-cba138c8ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:30.979049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8253b7-6df4-4064-ab0c-cba138c8ecb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:30.979049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8253b7-6df4-4064-ab0c-cba138c8ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:30.979049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8253b7-6df4-4064-ab0c-cba138c8ecb1 admin admin] PATH_INFO: `/v3/projects/07eed6f110004e0087bb116a82150093/users/4457504e0e9042d89baa49a602b0d65c/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:30.996967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8253b7-6df4-4064-ab0c-cba138c8ecb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07eed6f110004e0087bb116a82150093, user_id=4457504e0e9042d89baa49a602b0d65c, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:30.999656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8253b7-6df4-4064-ab0c-cba138c8ecb1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:31.104798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1094/2204] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:30 2022] PUT /identity/v3/projects/07eed6f110004e0087bb116a82150093/users/4457504e0e9042d89baa49a602b0d65c/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:36:31.108846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8253b7-6df4-4064-ab0c-cba138c8ecb1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:31.149859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754e6c649fd948e180d5681d23891a25'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:31.175168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754e6c649fd948e180d5681d23891a25'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:31.182585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754e6c649fd948e180d5681d23891a25'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:31.183310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754e6c649fd948e180d5681d23891a25'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:31.184057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-251ebb56-2c29-47b9-a591-6e852c9374b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754e6c649fd948e180d5681d23891a25'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:31.209113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8253b7-6df4-4064-ab0c-cba138c8ecb1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:31.217048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:31.217835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:31.218188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:31.218538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:31.219350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:31.221430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:31.240909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1095/2205] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:31.241639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:31.243605 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:31.260704 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4640314-b80a-4ef3-9e58-2e1a8a023848 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:31.265376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1111/2206] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:30 2022] DELETE /identity/v3/users/754e6c649fd948e180d5681d23891a25 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:31.268242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d464967-6bd7-4363-a98a-3ab30a7902fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:31.268242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d464967-6bd7-4363-a98a-3ab30a7902fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:31.268242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d464967-6bd7-4363-a98a-3ab30a7902fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:31.268242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d464967-6bd7-4363-a98a-3ab30a7902fc admin admin] PATH_INFO: `/v3/projects/07eed6f110004e0087bb116a82150093/users/4457504e0e9042d89baa49a602b0d65c/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:31.270271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c3ad3-da42-49b7-b48c-6242a43d189e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:31.270924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d464967-6bd7-4363-a98a-3ab30a7902fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=07eed6f110004e0087bb116a82150093, user_id=4457504e0e9042d89baa49a602b0d65c, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:31.271091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c3ad3-da42-49b7-b48c-6242a43d189e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:31.271492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c3ad3-da42-49b7-b48c-6242a43d189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:31.274108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d464967-6bd7-4363-a98a-3ab30a7902fc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:31.289803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1096/2207] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:31 2022] PUT /identity/v3/projects/07eed6f110004e0087bb116a82150093/users/4457504e0e9042d89baa49a602b0d65c/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:31.300819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-038c3ad3-da42-49b7-b48c-6242a43d189e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:31.300819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-038c3ad3-da42-49b7-b48c-6242a43d189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:31.303647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:31.304696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce65ab-642c-489f-b316-d31d03b22ab9 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:31.305191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce65ab-642c-489f-b316-d31d03b22ab9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:31.305649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce65ab-642c-489f-b316-d31d03b22ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:31.339913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-92ce65ab-642c-489f-b316-d31d03b22ab9 None None] MFA Rules not processed for user `4457504e0e9042d89baa49a602b0d65c`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:31.346067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-92ce65ab-642c-489f-b316-d31d03b22ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:31.367398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1112/2208] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:31.374256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c3ad3-da42-49b7-b48c-6242a43d189e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:31.376424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-038c3ad3-da42-49b7-b48c-6242a43d189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:31.422924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1097/2209] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:31.438628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c3ad3-da42-49b7-b48c-6242a43d189e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:31.443713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:31.444491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:31.444802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:31.445079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:31.458536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:31.458536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:31.461277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce65ab-642c-489f-b316-d31d03b22ab9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:31.462664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-92ce65ab-642c-489f-b316-d31d03b22ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:31.468793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1113/2210] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:31.524843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce65ab-642c-489f-b316-d31d03b22ab9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:31.528793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19c9ce-d014-4ea4-bb41-cd0b704a7c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:31.529454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c9ce-d014-4ea4-bb41-cd0b704a7c3b service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:31.529737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c9ce-d014-4ea4-bb41-cd0b704a7c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:31.530019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c9ce-d014-4ea4-bb41-cd0b704a7c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:31.531932 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ef19c9ce-d014-4ea4-bb41-cd0b704a7c3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:31.597737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:31.601361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19c9ce-d014-4ea4-bb41-cd0b704a7c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:31.603787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19c9ce-d014-4ea4-bb41-cd0b704a7c3b service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:31.613434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1098/2211] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:36:31.863610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:31.865757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:31.878780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b43b0-e1e0-494f-84e6-6315d689203a service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:31.884148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:31.885063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:31.885372 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:31.885655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] PATH_INFO: `/v3/projects/df49ddfe878c42a5aa0d01d00dbca834` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:31.886854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] RBAC: Authorizing `identity:delete_project(project_id=df49ddfe878c42a5aa0d01d00dbca834)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:31.890092 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:32.015247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df49ddfe878c42a5aa0d01d00dbca834', 'request_id': 'req-3b7b9296-a873-4813-b1ef-5b430fbce1be'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:32.081055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df49ddfe878c42a5aa0d01d00dbca834', 'request_id': 'req-3b7b9296-a873-4813-b1ef-5b430fbce1be'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:32.086929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df49ddfe878c42a5aa0d01d00dbca834', 'request_id': 'req-3b7b9296-a873-4813-b1ef-5b430fbce1be'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:32.098076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] The token cache is being invalidate because project df49ddfe878c42a5aa0d01d00dbca834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:32.098364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:32.098982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-3b7b9296-a873-4813-b1ef-5b430fbce1be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:32.100500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1114/2212] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:31 2022] DELETE /identity/v3/projects/df49ddfe878c42a5aa0d01d00dbca834 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:32.105625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ec4bc-56d3-4da9-a149-4b19609f054e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.106100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ec4bc-56d3-4da9-a149-4b19609f054e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.106207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ec4bc-56d3-4da9-a149-4b19609f054e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.125105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4d3ec4bc-56d3-4da9-a149-4b19609f054e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:32.126336 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4d3ec4bc-56d3-4da9-a149-4b19609f054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:32.187399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1099/2213] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:32.196821 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c201-eab9-4faf-9075-3c6e46200ae4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.197249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c201-eab9-4faf-9075-3c6e46200ae4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.197667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c201-eab9-4faf-9075-3c6e46200ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.222676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c812c201-eab9-4faf-9075-3c6e46200ae4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:32.223901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c812c201-eab9-4faf-9075-3c6e46200ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:32.232038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1115/2214] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:32.244449 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ec4bc-56d3-4da9-a149-4b19609f054e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:32.254410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ec4bc-56d3-4da9-a149-4b19609f054e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:32.260682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:32.262167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.262167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.262167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.262848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:32.263160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:32.264827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:32.289678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1100/2215] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:36:32.305755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812c201-eab9-4faf-9075-3c6e46200ae4 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:32.315717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812c201-eab9-4faf-9075-3c6e46200ae4 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:32.338629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ca1e2-7cdb-4af4-821d-cbb1642e8e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:32.339400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ca1e2-7cdb-4af4-821d-cbb1642e8e9f admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.340173 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ca1e2-7cdb-4af4-821d-cbb1642e8e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.340559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ca1e2-7cdb-4af4-821d-cbb1642e8e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.341371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ca1e2-7cdb-4af4-821d-cbb1642e8e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:32.343718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ca1e2-7cdb-4af4-821d-cbb1642e8e9f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:32.482179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1116/2216] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:32 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:32.486622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:32.500956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d364c2f-0b04-4eca-bd22-ff72c1a428b4 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:32.566410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183dd1c-ec2a-4f66-b7d7-d6fb30cdd49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:32.571572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183dd1c-ec2a-4f66-b7d7-d6fb30cdd49a admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.571572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183dd1c-ec2a-4f66-b7d7-d6fb30cdd49a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.571572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183dd1c-ec2a-4f66-b7d7-d6fb30cdd49a admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.571572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183dd1c-ec2a-4f66-b7d7-d6fb30cdd49a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:32.573023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183dd1c-ec2a-4f66-b7d7-d6fb30cdd49a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:32.674933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1101/2217] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:32 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:36:32.676131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ca1e2-7cdb-4af4-821d-cbb1642e8e9f admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:32.686729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ca1e2-7cdb-4af4-821d-cbb1642e8e9f admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:32.691507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254eb54-546a-4f70-a985-c5ae62230fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:32.692322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254eb54-546a-4f70-a985-c5ae62230fff admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.692636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254eb54-546a-4f70-a985-c5ae62230fff admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.692918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254eb54-546a-4f70-a985-c5ae62230fff admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.693695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254eb54-546a-4f70-a985-c5ae62230fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:32.695237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254eb54-546a-4f70-a985-c5ae62230fff admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:32.700783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:32.706613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1117/2218] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:32.713584 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183dd1c-ec2a-4f66-b7d7-d6fb30cdd49a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:32.725685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183dd1c-ec2a-4f66-b7d7-d6fb30cdd49a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:32.734294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b664c-f665-4218-83af-42d28de1966d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:32.734964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b664c-f665-4218-83af-42d28de1966d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.735254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b664c-f665-4218-83af-42d28de1966d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.735535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b664c-f665-4218-83af-42d28de1966d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.736272 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b664c-f665-4218-83af-42d28de1966d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:32.737903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b664c-f665-4218-83af-42d28de1966d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:32.749342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1102/2219] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:36:32.757754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254eb54-546a-4f70-a985-c5ae62230fff admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:32.767867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254eb54-546a-4f70-a985-c5ae62230fff admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:32.773201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:32.774165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.774654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.775038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] PATH_INFO: `/v3/projects/2b3305529bf940029d6cb600507cab3a/users/925cfe3e02964fcabeeb4b93f2081979/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.790334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3305529bf940029d6cb600507cab3a, user_id=925cfe3e02964fcabeeb4b93f2081979, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:32.793062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:32.875628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1118/2220] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:32 2022] PUT /identity/v3/projects/2b3305529bf940029d6cb600507cab3a/users/925cfe3e02964fcabeeb4b93f2081979/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:32.890998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722afb-b88b-4465-9b8d-bd4bc11fb9ac None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:32.892059 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722afb-b88b-4465-9b8d-bd4bc11fb9ac None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:32.892592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722afb-b88b-4465-9b8d-bd4bc11fb9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:32.910888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:32.933681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6d722afb-b88b-4465-9b8d-bd4bc11fb9ac None None] MFA Rules not processed for user `925cfe3e02964fcabeeb4b93f2081979`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:32.933681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6d722afb-b88b-4465-9b8d-bd4bc11fb9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:33.017191 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1103/2221] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:33.043130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:33.045091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:33.110114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cd091-6cc0-4252-bf34-af23c1013ef2 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:33.114012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ba85e-51ae-4f74-8fad-94706c5f3640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:33.114575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba85e-51ae-4f74-8fad-94706c5f3640 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:33.114936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba85e-51ae-4f74-8fad-94706c5f3640 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:33.115132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba85e-51ae-4f74-8fad-94706c5f3640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:33.123433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ba85e-51ae-4f74-8fad-94706c5f3640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:33.124894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ba85e-51ae-4f74-8fad-94706c5f3640 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:33.134250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1119/2222] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:34.415499 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d722afb-b88b-4465-9b8d-bd4bc11fb9ac None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:34.417173 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6d722afb-b88b-4465-9b8d-bd4bc11fb9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:34.485860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d722afb-b88b-4465-9b8d-bd4bc11fb9ac None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:34.492041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:34.492951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:34.493235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:34.493513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] PATH_INFO: `/v3/users/881f9aedfd644810ab368bae004716c6` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:34.494776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=881f9aedfd644810ab368bae004716c6)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:34.496737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:34.739128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '881f9aedfd644810ab368bae004716c6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:34.749250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '881f9aedfd644810ab368bae004716c6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:34.750049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '881f9aedfd644810ab368bae004716c6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:34.750668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '881f9aedfd644810ab368bae004716c6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:34.827714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '881f9aedfd644810ab368bae004716c6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:34.858226 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1104/2223] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:34 2022] DELETE /identity/v3/users/881f9aedfd644810ab368bae004716c6 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:34.858610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b38a-1060-419f-b6fc-65e48a950b77 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:34.858610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b38a-1060-419f-b6fc-65e48a950b77 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:34.858610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b38a-1060-419f-b6fc-65e48a950b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:34.873824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b585b38a-1060-419f-b6fc-65e48a950b77 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:34.875114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b585b38a-1060-419f-b6fc-65e48a950b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:34.936125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1120/2224] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:34.948481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:34.949951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:34.966739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:35.025955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990d62c-9de3-4450-9190-3c3de880d951 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:35.030426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:35.031150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:35.031545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:35.032042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:35.042163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:35.043787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:35.054728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1105/2225] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:35.321729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585b38a-1060-419f-b6fc-65e48a950b77 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:35.323676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b585b38a-1060-419f-b6fc-65e48a950b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:35.337166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585b38a-1060-419f-b6fc-65e48a950b77 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:35.343146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:35.344870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:35.345159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:35.345433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] PATH_INFO: `/v3/projects/a8d7918c87fc4feab324728c4739d326` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:35.346681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d7918c87fc4feab324728c4739d326)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:35.349279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:35.498175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d7918c87fc4feab324728c4739d326', 'request_id': 'req-785c75c6-9a02-42e5-b73a-5cec617bc24d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:35.604950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d7918c87fc4feab324728c4739d326', 'request_id': 'req-785c75c6-9a02-42e5-b73a-5cec617bc24d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:35.611022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d7918c87fc4feab324728c4739d326', 'request_id': 'req-785c75c6-9a02-42e5-b73a-5cec617bc24d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:35.622637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] The token cache is being invalidate because project a8d7918c87fc4feab324728c4739d326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:35.622942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:35.623538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:35.625093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1121/2226] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:35 2022] DELETE /identity/v3/projects/a8d7918c87fc4feab324728c4739d326 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:38.474817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:38.476252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:38.533814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776e37-ddc8-47c5-913b-6f5cc5ece998 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:38.606937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06e976-f883-4e2d-981e-673c14a614d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:38.607587 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06e976-f883-4e2d-981e-673c14a614d4 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:38.611068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06e976-f883-4e2d-981e-673c14a614d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:38.611068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06e976-f883-4e2d-981e-673c14a614d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:38.611068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7c06e976-f883-4e2d-981e-673c14a614d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:38.676361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06e976-f883-4e2d-981e-673c14a614d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:38.678401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06e976-f883-4e2d-981e-673c14a614d4 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:38.688967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1106/2227] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:39.613646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:39.618153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:39.621697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:39.702314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c75c6-9a02-42e5-b73a-5cec617bc24d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:39.706943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:39.707818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:39.708106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:39.708375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] PATH_INFO: `/v3/users/925cfe3e02964fcabeeb4b93f2081979` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:39.710157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=925cfe3e02964fcabeeb4b93f2081979)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:39.712077 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:39.893473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925cfe3e02964fcabeeb4b93f2081979'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:39.906097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925cfe3e02964fcabeeb4b93f2081979'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:39.913067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925cfe3e02964fcabeeb4b93f2081979'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:39.913800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925cfe3e02964fcabeeb4b93f2081979'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:39.914408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925cfe3e02964fcabeeb4b93f2081979'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:39.982269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1122/2228] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:39 2022] DELETE /identity/v3/users/925cfe3e02964fcabeeb4b93f2081979 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:39.987018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f195d-066a-4a11-b449-90dbb98bf9ab None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:39.987496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f195d-066a-4a11-b449-90dbb98bf9ab None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:39.987607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f195d-066a-4a11-b449-90dbb98bf9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:40.004875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d57f195d-066a-4a11-b449-90dbb98bf9ab None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:40.006592 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d57f195d-066a-4a11-b449-90dbb98bf9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:40.026184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:40.072237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1107/2229] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:40.087942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:40.089554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:40.165955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ca792-5973-4af5-b51f-ed1bb965a537 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:40.169753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f52b-b570-467a-8bdb-10b16f283267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:40.171768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f52b-b570-467a-8bdb-10b16f283267 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:40.176793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f52b-b570-467a-8bdb-10b16f283267 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:40.176793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f52b-b570-467a-8bdb-10b16f283267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:40.185737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f52b-b570-467a-8bdb-10b16f283267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:40.187611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f52b-b570-467a-8bdb-10b16f283267 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:40.201363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1123/2230] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:40.695432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f195d-066a-4a11-b449-90dbb98bf9ab None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:40.697070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d57f195d-066a-4a11-b449-90dbb98bf9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:40.712340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f195d-066a-4a11-b449-90dbb98bf9ab None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:40.717451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:40.724373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:40.724716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:40.725120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] PATH_INFO: `/v3/projects/2b3305529bf940029d6cb600507cab3a` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:40.728145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3305529bf940029d6cb600507cab3a)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:40.731695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:40.869316 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3305529bf940029d6cb600507cab3a', 'request_id': 'req-ef656350-d90b-403a-b1a4-94ae75c69f66'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:40.954826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3305529bf940029d6cb600507cab3a', 'request_id': 'req-ef656350-d90b-403a-b1a4-94ae75c69f66'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:40.961019 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3305529bf940029d6cb600507cab3a', 'request_id': 'req-ef656350-d90b-403a-b1a4-94ae75c69f66'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:40.972368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] The token cache is being invalidate because project 2b3305529bf940029d6cb600507cab3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:40.972699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:40.973347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-ef656350-d90b-403a-b1a4-94ae75c69f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:40.974907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1108/2231] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:40 2022] DELETE /identity/v3/projects/2b3305529bf940029d6cb600507cab3a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:40.979861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447a940-6467-46ce-8919-ccf59f903b89 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:40.980385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447a940-6467-46ce-8919-ccf59f903b89 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:40.980480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447a940-6467-46ce-8919-ccf59f903b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:40.999596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7447a940-6467-46ce-8919-ccf59f903b89 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:41.000917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7447a940-6467-46ce-8919-ccf59f903b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:41.020132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:41.059648 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1124/2232] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:41.068652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a440fb-1973-4cd1-9531-5a448ef92a52 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.069147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a440fb-1973-4cd1-9531-5a448ef92a52 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.069265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a440fb-1973-4cd1-9531-5a448ef92a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.088798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-52a440fb-1973-4cd1-9531-5a448ef92a52 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:41.090013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-52a440fb-1973-4cd1-9531-5a448ef92a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:41.097915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1109/2233] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:41.102032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447a940-6467-46ce-8919-ccf59f903b89 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:41.112196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447a940-6467-46ce-8919-ccf59f903b89 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:41.117737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:41.118379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.118664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.118941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.119663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:36:41.119968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:41.121658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:41.133785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1125/2234] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:36:41.137784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a440fb-1973-4cd1-9531-5a448ef92a52 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:41.146922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a440fb-1973-4cd1-9531-5a448ef92a52 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:41.152034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985329a-2d80-4f5b-86fc-f92b6f398b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:41.152711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985329a-2d80-4f5b-86fc-f92b6f398b02 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.152997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985329a-2d80-4f5b-86fc-f92b6f398b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.153271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985329a-2d80-4f5b-86fc-f92b6f398b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.153956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985329a-2d80-4f5b-86fc-f92b6f398b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:41.155827 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985329a-2d80-4f5b-86fc-f92b6f398b02 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:41.252239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1110/2235] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:36:41 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:41.256394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:41.266793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000e9f4-d044-46a9-b653-7675327f6f26 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:41.334900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3775f-53b6-440d-8435-0341c981525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:41.336480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3775f-53b6-440d-8435-0341c981525b admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.336859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3775f-53b6-440d-8435-0341c981525b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.337213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3775f-53b6-440d-8435-0341c981525b admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.337989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3775f-53b6-440d-8435-0341c981525b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:41.340101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3775f-53b6-440d-8435-0341c981525b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:41.464462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1126/2236] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:36:41 2022] POST /identity/v3/users => generated 291 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:36:41.468565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985329a-2d80-4f5b-86fc-f92b6f398b02 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:41.494182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985329a-2d80-4f5b-86fc-f92b6f398b02 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:41.499508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a0af-b0a3-4caa-9a87-ee3de985658e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:41.500210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a0af-b0a3-4caa-9a87-ee3de985658e admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.500642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a0af-b0a3-4caa-9a87-ee3de985658e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.500926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a0af-b0a3-4caa-9a87-ee3de985658e admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.501774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339a0af-b0a3-4caa-9a87-ee3de985658e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:41.503386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339a0af-b0a3-4caa-9a87-ee3de985658e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:41.515630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1111/2237] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:41.519587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3775f-53b6-440d-8435-0341c981525b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:41.530387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3775f-53b6-440d-8435-0341c981525b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:41.535381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d9ac-d7e0-4105-a682-904af00329c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:41.536040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d9ac-d7e0-4105-a682-904af00329c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.536324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d9ac-d7e0-4105-a682-904af00329c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.536701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d9ac-d7e0-4105-a682-904af00329c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.537762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80d9ac-d7e0-4105-a682-904af00329c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:41.539523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80d9ac-d7e0-4105-a682-904af00329c1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:41.549982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1127/2238] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:36:41.554712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a0af-b0a3-4caa-9a87-ee3de985658e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:41.564930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a0af-b0a3-4caa-9a87-ee3de985658e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:41.570050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:41.570907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.571177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.571454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] PATH_INFO: `/v3/projects/213787a3040e45efb2c14f5dd9dcf509/users/6294a11cbb2a475da4864ca15b7959d5/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.589722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213787a3040e45efb2c14f5dd9dcf509, user_id=6294a11cbb2a475da4864ca15b7959d5, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:41.589722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:41.733101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1112/2239] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:36:41 2022] PUT /identity/v3/projects/213787a3040e45efb2c14f5dd9dcf509/users/6294a11cbb2a475da4864ca15b7959d5/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:41.738446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e0581-a7e8-4bc6-b9a8-c741c9643179 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.738924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e0581-a7e8-4bc6-b9a8-c741c9643179 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.741911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e0581-a7e8-4bc6-b9a8-c741c9643179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.758548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b14e0581-a7e8-4bc6-b9a8-c741c9643179 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:41.759833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b14e0581-a7e8-4bc6-b9a8-c741c9643179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:41.827909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1128/2240] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:41.841849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:41.843914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:41.895639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:41.907544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c5b63-b848-46fd-9023-3364f1e8d0d7 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:41.911326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc94815-ef98-44d2-afb4-e69f1cd6ded9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:41.911982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc94815-ef98-44d2-afb4-e69f1cd6ded9 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:41.912269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc94815-ef98-44d2-afb4-e69f1cd6ded9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:41.912599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc94815-ef98-44d2-afb4-e69f1cd6ded9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:41.922426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc94815-ef98-44d2-afb4-e69f1cd6ded9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:41.923861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc94815-ef98-44d2-afb4-e69f1cd6ded9 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:41.933904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1113/2241] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:43.106066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380a553-7411-48ca-8a65-ac62beb7a030 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:43.106579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380a553-7411-48ca-8a65-ac62beb7a030 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:43.106775 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380a553-7411-48ca-8a65-ac62beb7a030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:43.137590 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1380a553-7411-48ca-8a65-ac62beb7a030 None None] MFA Rules not processed for user `052ae1db33014bb5915dc228f1b76d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:43.139156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1380a553-7411-48ca-8a65-ac62beb7a030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:43.184639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:43.227174 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1129/2242] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:43.777108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c4876-43a0-4417-b290-e683ab3b80cf None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:43.778710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c4876-43a0-4417-b290-e683ab3b80cf None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:43.779319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c4876-43a0-4417-b290-e683ab3b80cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:43.850199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f09c4876-43a0-4417-b290-e683ab3b80cf None None] MFA Rules not processed for user `6294a11cbb2a475da4864ca15b7959d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:43.857022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f09c4876-43a0-4417-b290-e683ab3b80cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:43.988697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1114/2243] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:44.003630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380a553-7411-48ca-8a65-ac62beb7a030 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:44.005094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1380a553-7411-48ca-8a65-ac62beb7a030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:44.072258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380a553-7411-48ca-8a65-ac62beb7a030 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:44.075911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5d869-41cb-4252-8c66-1ae2bb09de4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:44.076609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5d869-41cb-4252-8c66-1ae2bb09de4d service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:44.076905 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5d869-41cb-4252-8c66-1ae2bb09de4d service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:44.077184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5d869-41cb-4252-8c66-1ae2bb09de4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:44.087518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5d869-41cb-4252-8c66-1ae2bb09de4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:44.089316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5d869-41cb-4252-8c66-1ae2bb09de4d service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:44.103042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1130/2244] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:44.204510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c4876-43a0-4417-b290-e683ab3b80cf None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:44.206085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f09c4876-43a0-4417-b290-e683ab3b80cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:44.221802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c4876-43a0-4417-b290-e683ab3b80cf None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:44.227377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:44.228263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:44.228616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:44.228849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] PATH_INFO: `/v3/users/108d5002338544cf8afedeac10749a82` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:44.231553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=108d5002338544cf8afedeac10749a82)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:44.234409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:44.463894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108d5002338544cf8afedeac10749a82'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:44.486728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108d5002338544cf8afedeac10749a82'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:44.489420 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108d5002338544cf8afedeac10749a82'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:44.490278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108d5002338544cf8afedeac10749a82'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:44.587667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108d5002338544cf8afedeac10749a82'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:44.634234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa5ab5-4a84-4c6b-a18c-3cd0acae45fc None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:44.634781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa5ab5-4a84-4c6b-a18c-3cd0acae45fc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:44.634860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa5ab5-4a84-4c6b-a18c-3cd0acae45fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:44.649481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1115/2245] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:36:44 2022] DELETE /identity/v3/users/108d5002338544cf8afedeac10749a82 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:44.657349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7dfa5ab5-4a84-4c6b-a18c-3cd0acae45fc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:44.658707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7dfa5ab5-4a84-4c6b-a18c-3cd0acae45fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:44.734427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1131/2246] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:36:44.750423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:44.752103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:44.817735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:44.822547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e48b3-d712-41b4-be96-06ec311e88c0 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:44.826184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067cb61-a381-4b49-bc9b-746b83fff668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:44.826833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067cb61-a381-4b49-bc9b-746b83fff668 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:44.827123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067cb61-a381-4b49-bc9b-746b83fff668 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:44.827406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067cb61-a381-4b49-bc9b-746b83fff668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:44.837599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067cb61-a381-4b49-bc9b-746b83fff668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:44.839029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067cb61-a381-4b49-bc9b-746b83fff668 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:44.848989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1116/2247] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:36:45.409867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa5ab5-4a84-4c6b-a18c-3cd0acae45fc None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:36:45.411316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7dfa5ab5-4a84-4c6b-a18c-3cd0acae45fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:45.438525 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa5ab5-4a84-4c6b-a18c-3cd0acae45fc None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:36:45.445650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:36:45.447033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:45.447749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:45.448353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] PATH_INFO: `/v3/projects/6e6eaec274fd4b8295bf2a9fe68fbf26` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:45.449702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6eaec274fd4b8295bf2a9fe68fbf26)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:36:45.451794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:36:45.609242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6eaec274fd4b8295bf2a9fe68fbf26', 'request_id': 'req-10547385-a875-43c4-bebd-1b0447ee9105'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:45.690524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6eaec274fd4b8295bf2a9fe68fbf26', 'request_id': 'req-10547385-a875-43c4-bebd-1b0447ee9105'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:45.698783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6eaec274fd4b8295bf2a9fe68fbf26', 'request_id': 'req-10547385-a875-43c4-bebd-1b0447ee9105'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:45.713596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] The token cache is being invalidate because project 6e6eaec274fd4b8295bf2a9fe68fbf26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:36:45.713981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:45.714697 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-10547385-a875-43c4-bebd-1b0447ee9105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:36:45.716623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1132/2248] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:36:45 2022] DELETE /identity/v3/projects/6e6eaec274fd4b8295bf2a9fe68fbf26 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:36:46.013034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5b71f-6e97-485e-b0d8-d8ca498310b5 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:46.013611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5b71f-6e97-485e-b0d8-d8ca498310b5 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:46.013611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5b71f-6e97-485e-b0d8-d8ca498310b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:46.031598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-eff5b71f-6e97-485e-b0d8-d8ca498310b5 None None] MFA Rules not processed for user `6294a11cbb2a475da4864ca15b7959d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:46.032842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-eff5b71f-6e97-485e-b0d8-d8ca498310b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:46.091417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1117/2249] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:55.711971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:56.083634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=30 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:59.102678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b982aa-7738-496a-9546-91f3575035e2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:59.114725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b982aa-7738-496a-9546-91f3575035e2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:59.114725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b982aa-7738-496a-9546-91f3575035e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:59.122101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-52b982aa-7738-496a-9546-91f3575035e2 None None] MFA Rules not processed for user `c9dcfb252f0d44a280b3448094883e13`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:59.123300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-52b982aa-7738-496a-9546-91f3575035e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:59.185131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1133/2250] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:36:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:08.201610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5b71f-6e97-485e-b0d8-d8ca498310b5 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:08.204239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-eff5b71f-6e97-485e-b0d8-d8ca498310b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:08.387730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5b71f-6e97-485e-b0d8-d8ca498310b5 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:08.393753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:08.393753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:08.394000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:08.394078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] PATH_INFO: `/v3/users/b620f38c905a4240a665bf4de033396f` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:08.395450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b620f38c905a4240a665bf4de033396f)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:08.397460 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:08.441849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7356a-d4aa-4d8d-a8a4-c7c11a5b6c07 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:08.442283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7356a-d4aa-4d8d-a8a4-c7c11a5b6c07 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:08.442555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7356a-d4aa-4d8d-a8a4-c7c11a5b6c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:08.462191 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-66d7356a-d4aa-4d8d-a8a4-c7c11a5b6c07 None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:08.464079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-66d7356a-d4aa-4d8d-a8a4-c7c11a5b6c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:08.541720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1134/2251] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:37:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:37:08.559138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7356a-d4aa-4d8d-a8a4-c7c11a5b6c07 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:08.559138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-66d7356a-d4aa-4d8d-a8a4-c7c11a5b6c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:08.656898 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b620f38c905a4240a665bf4de033396f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:08.657497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7356a-d4aa-4d8d-a8a4-c7c11a5b6c07 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:08.672116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b620f38c905a4240a665bf4de033396f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:08.672770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee924e-fdc2-4d4e-a5e4-894dd957ac9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:08.672928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b620f38c905a4240a665bf4de033396f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:08.673622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b620f38c905a4240a665bf4de033396f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:08.673877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee924e-fdc2-4d4e-a5e4-894dd957ac9e service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:08.674459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee924e-fdc2-4d4e-a5e4-894dd957ac9e service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:08.675028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee924e-fdc2-4d4e-a5e4-894dd957ac9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:08.678710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f5ee924e-fdc2-4d4e-a5e4-894dd957ac9e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:08.692819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee924e-fdc2-4d4e-a5e4-894dd957ac9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:08.694491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee924e-fdc2-4d4e-a5e4-894dd957ac9e service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:08.703304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1135/2252] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:37:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:37:08.741904 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b620f38c905a4240a665bf4de033396f'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:08.760713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1118/2253] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:08 2022] DELETE /identity/v3/users/b620f38c905a4240a665bf4de033396f => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:08.765338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc799ae-53af-477a-9e62-27e0417521ff None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:08.765624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc799ae-53af-477a-9e62-27e0417521ff None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:08.765901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc799ae-53af-477a-9e62-27e0417521ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:08.793714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6dc799ae-53af-477a-9e62-27e0417521ff None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:08.794237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6dc799ae-53af-477a-9e62-27e0417521ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:08.806136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:08.855042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1136/2254] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:08.864871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:08.866352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:08.913120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:08.984911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a0ddc-bf32-4dde-adc4-4598d0c64afd admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:08.992798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fca562-6751-4815-aad9-6db9fa1722bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:08.993479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fca562-6751-4815-aad9-6db9fa1722bf service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:08.993767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fca562-6751-4815-aad9-6db9fa1722bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:08.995088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fca562-6751-4815-aad9-6db9fa1722bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:09.005134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fca562-6751-4815-aad9-6db9fa1722bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:09.006588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fca562-6751-4815-aad9-6db9fa1722bf service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:09.020566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1119/2255] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:09.805904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc799ae-53af-477a-9e62-27e0417521ff None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:09.810492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6dc799ae-53af-477a-9e62-27e0417521ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:09.840146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc799ae-53af-477a-9e62-27e0417521ff None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:09.857994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:09.858895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:09.859227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:09.859511 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] PATH_INFO: `/v3/projects/6062f9a0e12d4c4596ad9c1f243a72ca` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:09.861528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6062f9a0e12d4c4596ad9c1f243a72ca)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:09.868685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:10.030242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6062f9a0e12d4c4596ad9c1f243a72ca', 'request_id': 'req-1bdd982b-be13-4eda-999c-612a0edc44f7'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.183653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6062f9a0e12d4c4596ad9c1f243a72ca', 'request_id': 'req-1bdd982b-be13-4eda-999c-612a0edc44f7'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.207071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6062f9a0e12d4c4596ad9c1f243a72ca', 'request_id': 'req-1bdd982b-be13-4eda-999c-612a0edc44f7'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.283274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] The token cache is being invalidate because project 6062f9a0e12d4c4596ad9c1f243a72ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:10.283274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.283274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.283274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1137/2256] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:09 2022] DELETE /identity/v3/projects/6062f9a0e12d4c4596ad9c1f243a72ca => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:10.284359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fca562-6751-4815-aad9-6db9fa1722bf service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:10.297089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fca562-6751-4815-aad9-6db9fa1722bf service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:10.467436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:10.468761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:10.469493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:10.469981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] PATH_INFO: `/v3/users/4457504e0e9042d89baa49a602b0d65c` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:10.475515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4457504e0e9042d89baa49a602b0d65c)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:10.479335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:10.775989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4457504e0e9042d89baa49a602b0d65c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.786010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4457504e0e9042d89baa49a602b0d65c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.787031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4457504e0e9042d89baa49a602b0d65c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.787721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4457504e0e9042d89baa49a602b0d65c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.851755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4457504e0e9042d89baa49a602b0d65c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:10.899805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1120/2257] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:10 2022] DELETE /identity/v3/users/4457504e0e9042d89baa49a602b0d65c => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:11.040360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:11.043033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:11.069340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd982b-be13-4eda-999c-612a0edc44f7 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:11.174603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:11.175672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:11.176059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:11.176459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] PATH_INFO: `/v3/users/f8478d03a85d4c7b8c7d777ac4969b21` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:11.178026 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8478d03a85d4c7b8c7d777ac4969b21)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:11.180555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:11.397614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:11.404991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8478d03a85d4c7b8c7d777ac4969b21'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:11.437290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8478d03a85d4c7b8c7d777ac4969b21'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:11.458881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8478d03a85d4c7b8c7d777ac4969b21'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:11.461156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8478d03a85d4c7b8c7d777ac4969b21'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:11.461156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-64b3ba9e-b029-49d2-95b0-4fe70c3f10e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8478d03a85d4c7b8c7d777ac4969b21'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:11.567056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1138/2258] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:11 2022] DELETE /identity/v3/users/f8478d03a85d4c7b8c7d777ac4969b21 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:11.567576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:11.569472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:11.570811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa331-8819-4a73-b321-d72239239bad None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:11.571296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa331-8819-4a73-b321-d72239239bad None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:11.571652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa331-8819-4a73-b321-d72239239bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:11.610378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744d0fb-f0da-4a32-b915-971553c6958e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:11.617996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-ab4fa331-8819-4a73-b321-d72239239bad None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:11.624788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ab4fa331-8819-4a73-b321-d72239239bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:11.642881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:11.766751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:11.767969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:11.768376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:11.768721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] PATH_INFO: `/v3/projects/07eed6f110004e0087bb116a82150093` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:11.770156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] RBAC: Authorizing `identity:delete_project(project_id=07eed6f110004e0087bb116a82150093)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:11.772327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:11.777779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1139/2259] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:37:11.788016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fa331-8819-4a73-b321-d72239239bad None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:11.789692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ab4fa331-8819-4a73-b321-d72239239bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:11.946703 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fa331-8819-4a73-b321-d72239239bad None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:11.951808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07eed6f110004e0087bb116a82150093', 'request_id': 'req-d4de266c-7ef7-436f-98dd-47e2832d468d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:12.006739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07eed6f110004e0087bb116a82150093', 'request_id': 'req-d4de266c-7ef7-436f-98dd-47e2832d468d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:12.015358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07eed6f110004e0087bb116a82150093', 'request_id': 'req-d4de266c-7ef7-436f-98dd-47e2832d468d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:12.046486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] The token cache is being invalidate because project 07eed6f110004e0087bb116a82150093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:12.048984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:12.051759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d4de266c-7ef7-436f-98dd-47e2832d468d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:12.055618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c15261-04d8-4c07-bc38-1406a49e1452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:12.056369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c15261-04d8-4c07-bc38-1406a49e1452 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:12.057392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c15261-04d8-4c07-bc38-1406a49e1452 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:12.057681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c15261-04d8-4c07-bc38-1406a49e1452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:12.062343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-75c15261-04d8-4c07-bc38-1406a49e1452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:12.066358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1121/2260] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:11 2022] DELETE /identity/v3/projects/07eed6f110004e0087bb116a82150093 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:12.072004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69ec6f-9ad2-4a7a-8646-b7f9030e1efe None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:12.072559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69ec6f-9ad2-4a7a-8646-b7f9030e1efe None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:12.073251 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69ec6f-9ad2-4a7a-8646-b7f9030e1efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:12.170385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-3c69ec6f-9ad2-4a7a-8646-b7f9030e1efe None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:12.171806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3c69ec6f-9ad2-4a7a-8646-b7f9030e1efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:12.218132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c15261-04d8-4c07-bc38-1406a49e1452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:12.222044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c15261-04d8-4c07-bc38-1406a49e1452 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:12.247070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1140/2261] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:12.252070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:12.318529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1122/2262] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:12.327708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf1da5-68af-4cb8-94df-9f7715ee9e5e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:12.328498 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf1da5-68af-4cb8-94df-9f7715ee9e5e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:12.328967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf1da5-68af-4cb8-94df-9f7715ee9e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:12.356503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-dbaf1da5-68af-4cb8-94df-9f7715ee9e5e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:12.361424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dbaf1da5-68af-4cb8-94df-9f7715ee9e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:12.377429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1141/2263] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:12.385171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69ec6f-9ad2-4a7a-8646-b7f9030e1efe None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:12.420775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69ec6f-9ad2-4a7a-8646-b7f9030e1efe None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:12.445225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:12.445914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:12.446199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:12.451090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:12.451090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:37:12.451090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:12.451090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:12.483570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf1da5-68af-4cb8-94df-9f7715ee9e5e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:12.485442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1123/2264] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:37:12.495297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf1da5-68af-4cb8-94df-9f7715ee9e5e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:12.506415 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04bfe3-2dd9-4a4e-b8b8-e8ef235dee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:12.508327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04bfe3-2dd9-4a4e-b8b8-e8ef235dee7a admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:12.508327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04bfe3-2dd9-4a4e-b8b8-e8ef235dee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:12.508327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04bfe3-2dd9-4a4e-b8b8-e8ef235dee7a admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:12.508327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04bfe3-2dd9-4a4e-b8b8-e8ef235dee7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:12.514467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04bfe3-2dd9-4a4e-b8b8-e8ef235dee7a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:12.630418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1142/2265] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:12 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:37:12.635523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:12.651396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609e69-6ba1-475b-9fed-5b416e36d9c1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:12.708885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:12.802690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ab487-3c71-4b08-85b9-9b1a16201854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:12.803488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab487-3c71-4b08-85b9-9b1a16201854 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:12.803838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab487-3c71-4b08-85b9-9b1a16201854 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:12.804180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab487-3c71-4b08-85b9-9b1a16201854 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:12.805056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ab487-3c71-4b08-85b9-9b1a16201854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:12.807149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ab487-3c71-4b08-85b9-9b1a16201854 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:12.955085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1124/2266] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:12 2022] POST /identity/v3/users => generated 287 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:12.956291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04bfe3-2dd9-4a4e-b8b8-e8ef235dee7a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:12.971909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04bfe3-2dd9-4a4e-b8b8-e8ef235dee7a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:12.980071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fa39b-259d-46bf-806d-0ca6ae834042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:13.017650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fa39b-259d-46bf-806d-0ca6ae834042 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:13.017650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fa39b-259d-46bf-806d-0ca6ae834042 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:13.017650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fa39b-259d-46bf-806d-0ca6ae834042 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:13.017650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fa39b-259d-46bf-806d-0ca6ae834042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:13.017650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fa39b-259d-46bf-806d-0ca6ae834042 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:13.017650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1143/2267] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:13.019376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ab487-3c71-4b08-85b9-9b1a16201854 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:13.030868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ab487-3c71-4b08-85b9-9b1a16201854 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:13.036394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:13.037435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:13.037799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:13.038150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:13.038964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:13.040825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:13.056622 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1125/2268] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:13.058516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fa39b-259d-46bf-806d-0ca6ae834042 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:13.078784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fa39b-259d-46bf-806d-0ca6ae834042 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:13.083674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd3755-6710-4c83-8eee-50b5bb6e6c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:13.084615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd3755-6710-4c83-8eee-50b5bb6e6c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:13.085236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd3755-6710-4c83-8eee-50b5bb6e6c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:13.085762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd3755-6710-4c83-8eee-50b5bb6e6c96 admin admin] PATH_INFO: `/v3/projects/a8c8c9bbcf144dcd86d0b3fb25d33c6b/users/6ff52d1f15fb441f865e8f31973fb981/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:13.102080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd3755-6710-4c83-8eee-50b5bb6e6c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c8c9bbcf144dcd86d0b3fb25d33c6b, user_id=6ff52d1f15fb441f865e8f31973fb981, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:13.120998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd3755-6710-4c83-8eee-50b5bb6e6c96 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:13.283215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:13.285639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:13.290180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1144/2269] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:13 2022] PUT /identity/v3/projects/a8c8c9bbcf144dcd86d0b3fb25d33c6b/users/6ff52d1f15fb441f865e8f31973fb981/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:13.291692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a7dd7-aff1-45c3-a3e4-1194a4e42764 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:13.291981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a7dd7-aff1-45c3-a3e4-1194a4e42764 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:13.292256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a7dd7-aff1-45c3-a3e4-1194a4e42764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:13.301180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21a1bf-3dd6-4e82-a408-69c4fde8fd71 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:13.344633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5a1a7dd7-aff1-45c3-a3e4-1194a4e42764 None None] MFA Rules not processed for user `6ff52d1f15fb441f865e8f31973fb981`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:13.347372 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5a1a7dd7-aff1-45c3-a3e4-1194a4e42764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:13.377924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:13.378777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:13.379064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:13.379347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] PATH_INFO: `/v3/projects/1601ada8906247f2bb4bd21f78517042` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:13.380623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1601ada8906247f2bb4bd21f78517042)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:13.382555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:13.512804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1145/2270] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:13.517071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a7dd7-aff1-45c3-a3e4-1194a4e42764 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:13.548315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a7dd7-aff1-45c3-a3e4-1194a4e42764 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:13.573771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1601ada8906247f2bb4bd21f78517042', 'request_id': 'req-e3d96801-f283-4e6b-a844-1318c5d1d53b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:13.689323 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:13.691954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1601ada8906247f2bb4bd21f78517042', 'request_id': 'req-e3d96801-f283-4e6b-a844-1318c5d1d53b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:13.700441 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1601ada8906247f2bb4bd21f78517042', 'request_id': 'req-e3d96801-f283-4e6b-a844-1318c5d1d53b'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:13.708508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b82c42-a14b-4447-960a-5cffa7b429a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:13.710081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b82c42-a14b-4447-960a-5cffa7b429a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:13.711524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b82c42-a14b-4447-960a-5cffa7b429a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:13.721039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b82c42-a14b-4447-960a-5cffa7b429a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:13.721802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b82c42-a14b-4447-960a-5cffa7b429a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:13.729763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b82c42-a14b-4447-960a-5cffa7b429a9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:13.730122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] The token cache is being invalidate because project 1601ada8906247f2bb4bd21f78517042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:13.730122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:13.730122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:13.749221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1126/2271] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:13 2022] DELETE /identity/v3/projects/1601ada8906247f2bb4bd21f78517042 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:13.846148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1146/2272] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:13 2022] POST /identity/v3/projects => generated 373 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:13.854594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:13.858067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:13.899000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d96801-f283-4e6b-a844-1318c5d1d53b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:13.961407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:14.002338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0201b0-444b-4659-9d36-27c2eda76693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:14.005784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0201b0-444b-4659-9d36-27c2eda76693 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:14.005784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0201b0-444b-4659-9d36-27c2eda76693 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:14.005784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0201b0-444b-4659-9d36-27c2eda76693 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:14.005784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0201b0-444b-4659-9d36-27c2eda76693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:14.006414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0201b0-444b-4659-9d36-27c2eda76693 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:14.122871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1127/2273] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:13 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:14.127078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b82c42-a14b-4447-960a-5cffa7b429a9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:14.138568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b82c42-a14b-4447-960a-5cffa7b429a9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:14.144225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff25fd-d7a1-4c32-beab-7a133d24f3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:14.144946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff25fd-d7a1-4c32-beab-7a133d24f3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:14.145292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff25fd-d7a1-4c32-beab-7a133d24f3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:14.145577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff25fd-d7a1-4c32-beab-7a133d24f3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:14.146403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff25fd-d7a1-4c32-beab-7a133d24f3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:14.148024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff25fd-d7a1-4c32-beab-7a133d24f3b1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:14.184876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1147/2274] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:14.186056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0201b0-444b-4659-9d36-27c2eda76693 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:14.205484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0201b0-444b-4659-9d36-27c2eda76693 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:14.217365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4135d-2143-4b4c-a0b2-87d599421ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:14.218474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4135d-2143-4b4c-a0b2-87d599421ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:14.220940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4135d-2143-4b4c-a0b2-87d599421ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:14.221241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4135d-2143-4b4c-a0b2-87d599421ffe admin admin] PATH_INFO: `/v3/projects/b1a1c667f3cf4d6b81912a1c06268127/users/e35cb2c80bec46afb8244bfe291edd41/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:14.280843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4135d-2143-4b4c-a0b2-87d599421ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a1c667f3cf4d6b81912a1c06268127, user_id=e35cb2c80bec46afb8244bfe291edd41, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:14.287008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4135d-2143-4b4c-a0b2-87d599421ffe admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:14.413480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1128/2275] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:14 2022] PUT /identity/v3/projects/b1a1c667f3cf4d6b81912a1c06268127/users/e35cb2c80bec46afb8244bfe291edd41/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:14.414282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff25fd-d7a1-4c32-beab-7a133d24f3b1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:14.435324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff25fd-d7a1-4c32-beab-7a133d24f3b1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:14.531973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef777c50-9420-4862-992e-bcfeace3de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:14.533085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef777c50-9420-4862-992e-bcfeace3de2e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:14.533565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef777c50-9420-4862-992e-bcfeace3de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:14.533992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef777c50-9420-4862-992e-bcfeace3de2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:14.535028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef777c50-9420-4862-992e-bcfeace3de2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:14.536929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef777c50-9420-4862-992e-bcfeace3de2e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:14.551058 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1148/2276] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:14.554346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4135d-2143-4b4c-a0b2-87d599421ffe admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:14.568780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4135d-2143-4b4c-a0b2-87d599421ffe admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:14.575596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:14.580727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:14.580727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:14.580727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] PATH_INFO: `/v3/projects/b1a1c667f3cf4d6b81912a1c06268127/users/e35cb2c80bec46afb8244bfe291edd41/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:14.583528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a1c667f3cf4d6b81912a1c06268127, user_id=e35cb2c80bec46afb8244bfe291edd41, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:14.587280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:14.606506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1129/2277] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:14 2022] PUT /identity/v3/projects/b1a1c667f3cf4d6b81912a1c06268127/users/e35cb2c80bec46afb8244bfe291edd41/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:14.613899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033c65-fa60-434f-afce-994faa8568ad None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:14.614206 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033c65-fa60-434f-afce-994faa8568ad None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:14.614480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033c65-fa60-434f-afce-994faa8568ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:14.654978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4d033c65-fa60-434f-afce-994faa8568ad None None] MFA Rules not processed for user `e35cb2c80bec46afb8244bfe291edd41`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:14.656116 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4d033c65-fa60-434f-afce-994faa8568ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:14.768847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1149/2278] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:37:14.787551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:14.789796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:14.896848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142e886-a9be-4fad-be93-2d586fb2d061 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:14.909231 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ec481-8dd8-46f3-b8b0-f5f09aa14362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:14.909984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ec481-8dd8-46f3-b8b0-f5f09aa14362 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:14.910541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ec481-8dd8-46f3-b8b0-f5f09aa14362 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:14.911137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ec481-8dd8-46f3-b8b0-f5f09aa14362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:14.915924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f69ec481-8dd8-46f3-b8b0-f5f09aa14362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:15.024588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ec481-8dd8-46f3-b8b0-f5f09aa14362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:15.030193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ec481-8dd8-46f3-b8b0-f5f09aa14362 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:15.037198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1130/2279] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:18.638485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d033c65-fa60-434f-afce-994faa8568ad None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:18.697816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:18.726931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d033c65-fa60-434f-afce-994faa8568ad None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:18.732456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:18.733509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.736783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.736783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] PATH_INFO: `/v3/users/6ff52d1f15fb441f865e8f31973fb981` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.736783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff52d1f15fb441f865e8f31973fb981)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:18.738143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:18.985545 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff52d1f15fb441f865e8f31973fb981'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.007317 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff52d1f15fb441f865e8f31973fb981'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.016601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff52d1f15fb441f865e8f31973fb981'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.017369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff52d1f15fb441f865e8f31973fb981'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.017989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff52d1f15fb441f865e8f31973fb981'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.101857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1150/2280] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:18 2022] DELETE /identity/v3/users/6ff52d1f15fb441f865e8f31973fb981 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:19.106447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d4f72-c8d3-4b9f-ad23-090574f7e741 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:19.106447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d4f72-c8d3-4b9f-ad23-090574f7e741 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:19.106447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d4f72-c8d3-4b9f-ad23-090574f7e741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:19.125814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e86d4f72-c8d3-4b9f-ad23-090574f7e741 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:19.127144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e86d4f72-c8d3-4b9f-ad23-090574f7e741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:19.189606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:19.237333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1131/2281] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:19.248870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:19.250729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:19.338931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cb886-3519-4518-8cb2-530110e64016 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:19.343356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093fba9-27a3-42df-a3c9-b9eaf51f7902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:19.344108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093fba9-27a3-42df-a3c9-b9eaf51f7902 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:19.344469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093fba9-27a3-42df-a3c9-b9eaf51f7902 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:19.344848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093fba9-27a3-42df-a3c9-b9eaf51f7902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:19.356257 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093fba9-27a3-42df-a3c9-b9eaf51f7902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:19.358002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093fba9-27a3-42df-a3c9-b9eaf51f7902 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:19.372335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1151/2282] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:37:20.334684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d4f72-c8d3-4b9f-ad23-090574f7e741 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.338514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e86d4f72-c8d3-4b9f-ad23-090574f7e741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:20.354183 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d4f72-c8d3-4b9f-ad23-090574f7e741 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.361643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.363915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.364408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.365027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] PATH_INFO: `/v3/projects/a8c8c9bbcf144dcd86d0b3fb25d33c6b` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.366905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c8c9bbcf144dcd86d0b3fb25d33c6b)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.370124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.535044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c8c9bbcf144dcd86d0b3fb25d33c6b', 'request_id': 'req-4092d8f2-4c99-4287-bff7-fd0112cb1898'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.630763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c8c9bbcf144dcd86d0b3fb25d33c6b', 'request_id': 'req-4092d8f2-4c99-4287-bff7-fd0112cb1898'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.639156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c8c9bbcf144dcd86d0b3fb25d33c6b', 'request_id': 'req-4092d8f2-4c99-4287-bff7-fd0112cb1898'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.658090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] The token cache is being invalidate because project a8c8c9bbcf144dcd86d0b3fb25d33c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:20.658699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.660044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.665777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1132/2283] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:20 2022] DELETE /identity/v3/projects/a8c8c9bbcf144dcd86d0b3fb25d33c6b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:20.670678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093fba9-27a3-42df-a3c9-b9eaf51f7902 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.682693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093fba9-27a3-42df-a3c9-b9eaf51f7902 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.834839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.835760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.836051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.836335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] PATH_INFO: `/v3/users/e35cb2c80bec46afb8244bfe291edd41` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.837607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35cb2c80bec46afb8244bfe291edd41)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.840804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.857178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:21.429417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35cb2c80bec46afb8244bfe291edd41'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:21.449097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35cb2c80bec46afb8244bfe291edd41'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:21.459793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35cb2c80bec46afb8244bfe291edd41'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:21.460670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35cb2c80bec46afb8244bfe291edd41'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:21.461541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a7b4f605-dc97-4dd4-aac0-8005a0975706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35cb2c80bec46afb8244bfe291edd41'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:21.560564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1152/2284] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:20 2022] DELETE /identity/v3/users/e35cb2c80bec46afb8244bfe291edd41 => generated 0 bytes in 894 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:22.078406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:22.078406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:22.090899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d8f2-4c99-4287-bff7-fd0112cb1898 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:22.171235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:22.172102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:22.172391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:22.172707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] PATH_INFO: `/v3/projects/b1a1c667f3cf4d6b81912a1c06268127` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:22.173913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a1c667f3cf4d6b81912a1c06268127)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:22.175808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:22.386083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a1c667f3cf4d6b81912a1c06268127', 'request_id': 'req-fff9d704-9714-486e-88b3-e4d621437297'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:22.445847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a1c667f3cf4d6b81912a1c06268127', 'request_id': 'req-fff9d704-9714-486e-88b3-e4d621437297'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:22.454111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:22.457394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a1c667f3cf4d6b81912a1c06268127', 'request_id': 'req-fff9d704-9714-486e-88b3-e4d621437297'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:22.470867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] The token cache is being invalidate because project b1a1c667f3cf4d6b81912a1c06268127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:22.471171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:22.471844 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-fff9d704-9714-486e-88b3-e4d621437297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:22.473465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1133/2285] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:22 2022] DELETE /identity/v3/projects/b1a1c667f3cf4d6b81912a1c06268127 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:22.481584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc038-9111-49ac-a5f6-642cee5fea44 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:22.481584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc038-9111-49ac-a5f6-642cee5fea44 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:22.481584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc038-9111-49ac-a5f6-642cee5fea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:22.513567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-21ccc038-9111-49ac-a5f6-642cee5fea44 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:22.514831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-21ccc038-9111-49ac-a5f6-642cee5fea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:22.578621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1153/2286] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:22.586230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ddf91-7dad-49e0-af4c-1ec9d3b79ed8 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:22.586521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ddf91-7dad-49e0-af4c-1ec9d3b79ed8 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:22.586808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ddf91-7dad-49e0-af4c-1ec9d3b79ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:22.605455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6b5ddf91-7dad-49e0-af4c-1ec9d3b79ed8 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:22.606640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6b5ddf91-7dad-49e0-af4c-1ec9d3b79ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:22.614517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1134/2287] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:22.618766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccc038-9111-49ac-a5f6-642cee5fea44 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:22.629047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccc038-9111-49ac-a5f6-642cee5fea44 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:22.633745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:22.634400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:22.635221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:22.635504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:22.636261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:37:22.636585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:22.638171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:22.651223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1154/2288] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:37:22.656271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ddf91-7dad-49e0-af4c-1ec9d3b79ed8 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:22.667825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ddf91-7dad-49e0-af4c-1ec9d3b79ed8 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:22.672387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900408a-42dd-4a8f-a538-80e34a189472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:22.673089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900408a-42dd-4a8f-a538-80e34a189472 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:22.673544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900408a-42dd-4a8f-a538-80e34a189472 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:22.673860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900408a-42dd-4a8f-a538-80e34a189472 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:22.674567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900408a-42dd-4a8f-a538-80e34a189472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:22.676468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900408a-42dd-4a8f-a538-80e34a189472 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:22.780859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1135/2289] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:22 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:37:22.782175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:22.792038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0a814-3246-464e-80a3-2e42457bbe85 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:22.848483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0393a97-5620-401c-87ba-84af1d374ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:22.849109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0393a97-5620-401c-87ba-84af1d374ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:22.849394 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0393a97-5620-401c-87ba-84af1d374ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:22.849702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0393a97-5620-401c-87ba-84af1d374ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:22.850406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0393a97-5620-401c-87ba-84af1d374ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:22.852322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0393a97-5620-401c-87ba-84af1d374ed2 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:22.934380 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:22.943419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1155/2290] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:22 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:37:22.948310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900408a-42dd-4a8f-a538-80e34a189472 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:22.958919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900408a-42dd-4a8f-a538-80e34a189472 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:22.963529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd4d6-e2a2-4218-85cc-16d233ff53eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:22.964181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd4d6-e2a2-4218-85cc-16d233ff53eb admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:22.964463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd4d6-e2a2-4218-85cc-16d233ff53eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:22.964767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd4d6-e2a2-4218-85cc-16d233ff53eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:22.965493 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd4d6-e2a2-4218-85cc-16d233ff53eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:22.967073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd4d6-e2a2-4218-85cc-16d233ff53eb admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:22.977502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1136/2291] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:22.981378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0393a97-5620-401c-87ba-84af1d374ed2 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:22.991481 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0393a97-5620-401c-87ba-84af1d374ed2 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:22.997309 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2961d-24b6-4ffd-91e8-af9556d97919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:22.998002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2961d-24b6-4ffd-91e8-af9556d97919 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.003115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2961d-24b6-4ffd-91e8-af9556d97919 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.003115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2961d-24b6-4ffd-91e8-af9556d97919 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.003115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2961d-24b6-4ffd-91e8-af9556d97919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:23.003115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2961d-24b6-4ffd-91e8-af9556d97919 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:23.012387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1156/2292] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:23.016220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd4d6-e2a2-4218-85cc-16d233ff53eb admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:23.027097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd4d6-e2a2-4218-85cc-16d233ff53eb admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:23.032634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821fbd-41e1-42be-a977-993cb3d8e386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:23.033535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fbd-41e1-42be-a977-993cb3d8e386 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.033821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fbd-41e1-42be-a977-993cb3d8e386 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.034103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fbd-41e1-42be-a977-993cb3d8e386 admin admin] PATH_INFO: `/v3/projects/39dfe24befeb4c568c343835b11624a4/users/af854687c48d437ba86ab3c7057d2750/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.049214 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b821fbd-41e1-42be-a977-993cb3d8e386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39dfe24befeb4c568c343835b11624a4, user_id=af854687c48d437ba86ab3c7057d2750, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:23.051848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b821fbd-41e1-42be-a977-993cb3d8e386 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:23.131772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1137/2293] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:23 2022] PUT /identity/v3/projects/39dfe24befeb4c568c343835b11624a4/users/af854687c48d437ba86ab3c7057d2750/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:23.141908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff0a9e-00a2-4f61-be20-472a0630f822 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.145829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff0a9e-00a2-4f61-be20-472a0630f822 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.145829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff0a9e-00a2-4f61-be20-472a0630f822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.170193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-43ff0a9e-00a2-4f61-be20-472a0630f822 None None] MFA Rules not processed for user `af854687c48d437ba86ab3c7057d2750`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:23.171463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-43ff0a9e-00a2-4f61-be20-472a0630f822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:23.236938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1157/2294] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:23.241478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821fbd-41e1-42be-a977-993cb3d8e386 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:23.308568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821fbd-41e1-42be-a977-993cb3d8e386 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:23.313267 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2c8f3-8096-44bc-91fb-89247fa7e757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:23.313930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2c8f3-8096-44bc-91fb-89247fa7e757 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.314219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2c8f3-8096-44bc-91fb-89247fa7e757 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.314501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2c8f3-8096-44bc-91fb-89247fa7e757 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.315491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2c8f3-8096-44bc-91fb-89247fa7e757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:23.317436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2c8f3-8096-44bc-91fb-89247fa7e757 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:23.413151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1138/2295] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:23 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:23.417818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff0a9e-00a2-4f61-be20-472a0630f822 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:23.428784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff0a9e-00a2-4f61-be20-472a0630f822 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:23.495956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd0ce9-efcf-4aea-93b1-9eb754b582d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:23.496670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd0ce9-efcf-4aea-93b1-9eb754b582d0 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.496935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd0ce9-efcf-4aea-93b1-9eb754b582d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.497214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd0ce9-efcf-4aea-93b1-9eb754b582d0 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.497910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd0ce9-efcf-4aea-93b1-9eb754b582d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:23.499823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd0ce9-efcf-4aea-93b1-9eb754b582d0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:23.590799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1158/2296] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:23 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:23.597351 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2c8f3-8096-44bc-91fb-89247fa7e757 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:23.619227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2c8f3-8096-44bc-91fb-89247fa7e757 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:23.624225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575295df-4396-4b37-bd5c-6acce30bcdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:23.624938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575295df-4396-4b37-bd5c-6acce30bcdc8 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.625229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575295df-4396-4b37-bd5c-6acce30bcdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.625514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575295df-4396-4b37-bd5c-6acce30bcdc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.626284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575295df-4396-4b37-bd5c-6acce30bcdc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:23.627897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575295df-4396-4b37-bd5c-6acce30bcdc8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:23.639211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1139/2297] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:23.643287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd0ce9-efcf-4aea-93b1-9eb754b582d0 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:23.654564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd0ce9-efcf-4aea-93b1-9eb754b582d0 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:23.659515 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc6937-e7a5-45a7-97e0-5ec2fbf8bee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:23.660278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc6937-e7a5-45a7-97e0-5ec2fbf8bee3 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.660676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc6937-e7a5-45a7-97e0-5ec2fbf8bee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.661037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc6937-e7a5-45a7-97e0-5ec2fbf8bee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.661849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc6937-e7a5-45a7-97e0-5ec2fbf8bee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:23.663575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc6937-e7a5-45a7-97e0-5ec2fbf8bee3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:23.676946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1159/2298] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:23.679235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575295df-4396-4b37-bd5c-6acce30bcdc8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:23.690529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575295df-4396-4b37-bd5c-6acce30bcdc8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:23.695260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96fb0f-68d6-4f98-8b7c-0b3a12057e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:23.696162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96fb0f-68d6-4f98-8b7c-0b3a12057e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.696447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96fb0f-68d6-4f98-8b7c-0b3a12057e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.696760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96fb0f-68d6-4f98-8b7c-0b3a12057e76 admin admin] PATH_INFO: `/v3/projects/dd46c6dc4d414dcc9f2c6b12f8e2a115/users/358b6b799ad748429cd5b093848c50e4/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.711924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96fb0f-68d6-4f98-8b7c-0b3a12057e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd46c6dc4d414dcc9f2c6b12f8e2a115, user_id=358b6b799ad748429cd5b093848c50e4, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:23.715126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96fb0f-68d6-4f98-8b7c-0b3a12057e76 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:23.766244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1140/2299] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:23 2022] PUT /identity/v3/projects/dd46c6dc4d414dcc9f2c6b12f8e2a115/users/358b6b799ad748429cd5b093848c50e4/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:23.781647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd115db4-3abf-4a96-9b14-3c974b7133da None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.781943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd115db4-3abf-4a96-9b14-3c974b7133da None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.782270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd115db4-3abf-4a96-9b14-3c974b7133da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.818907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-dd115db4-3abf-4a96-9b14-3c974b7133da None None] MFA Rules not processed for user `358b6b799ad748429cd5b093848c50e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:23.820384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dd115db4-3abf-4a96-9b14-3c974b7133da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:23.866663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:23.890509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1160/2300] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:23.894546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96fb0f-68d6-4f98-8b7c-0b3a12057e76 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:23.937425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:23.968486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96fb0f-68d6-4f98-8b7c-0b3a12057e76 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:23.968486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66e878-923a-4cfa-88df-87f206d63902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:23.969077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66e878-923a-4cfa-88df-87f206d63902 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:23.969361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66e878-923a-4cfa-88df-87f206d63902 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:23.969640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66e878-923a-4cfa-88df-87f206d63902 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:23.970378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66e878-923a-4cfa-88df-87f206d63902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:23.972286 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66e878-923a-4cfa-88df-87f206d63902 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:24.080748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1141/2301] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:23 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:24.084840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd115db4-3abf-4a96-9b14-3c974b7133da None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:24.101527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd115db4-3abf-4a96-9b14-3c974b7133da None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:24.161221 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4db374-4a5b-4c32-a7bc-b858c4e724b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:24.161896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4db374-4a5b-4c32-a7bc-b858c4e724b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:24.162183 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4db374-4a5b-4c32-a7bc-b858c4e724b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:24.162461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4db374-4a5b-4c32-a7bc-b858c4e724b4 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:24.164685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4db374-4a5b-4c32-a7bc-b858c4e724b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:24.165148 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4db374-4a5b-4c32-a7bc-b858c4e724b4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:24.247505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1161/2302] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:24 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:24.251434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66e878-923a-4cfa-88df-87f206d63902 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:24.265728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66e878-923a-4cfa-88df-87f206d63902 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:24.270472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a00da-d63d-4222-a452-09bf9f3455c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:24.271144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a00da-d63d-4222-a452-09bf9f3455c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:24.271458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a00da-d63d-4222-a452-09bf9f3455c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:24.271750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a00da-d63d-4222-a452-09bf9f3455c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:24.272482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a00da-d63d-4222-a452-09bf9f3455c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:24.274103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a00da-d63d-4222-a452-09bf9f3455c7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:24.284833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1142/2303] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:24.289239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4db374-4a5b-4c32-a7bc-b858c4e724b4 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:24.299284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4db374-4a5b-4c32-a7bc-b858c4e724b4 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:24.303822 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73403d4-6849-453c-ba57-86099276dcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:24.304722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73403d4-6849-453c-ba57-86099276dcaa admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:24.305007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73403d4-6849-453c-ba57-86099276dcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:24.305288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73403d4-6849-453c-ba57-86099276dcaa admin admin] PATH_INFO: `/v3/projects/30f30138096346329ab8d2a6c0db5e41/users/945a2dd8428e4edba616f1e3d7951921/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:24.320290 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73403d4-6849-453c-ba57-86099276dcaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f30138096346329ab8d2a6c0db5e41, user_id=945a2dd8428e4edba616f1e3d7951921, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:24.323562 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73403d4-6849-453c-ba57-86099276dcaa admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:24.389859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1162/2304] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:24 2022] PUT /identity/v3/projects/30f30138096346329ab8d2a6c0db5e41/users/945a2dd8428e4edba616f1e3d7951921/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:24.393716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a00da-d63d-4222-a452-09bf9f3455c7 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:24.403527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a00da-d63d-4222-a452-09bf9f3455c7 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:24.468407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067ac73-1c1f-4e50-846a-c4a94634b3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:24.469033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067ac73-1c1f-4e50-846a-c4a94634b3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:24.469318 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067ac73-1c1f-4e50-846a-c4a94634b3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:24.469594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067ac73-1c1f-4e50-846a-c4a94634b3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:24.470330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067ac73-1c1f-4e50-846a-c4a94634b3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:24.471926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067ac73-1c1f-4e50-846a-c4a94634b3b1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:24.482707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1143/2305] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:24.487234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73403d4-6849-453c-ba57-86099276dcaa admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:24.503843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73403d4-6849-453c-ba57-86099276dcaa admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:24.508817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:24.510306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:24.510596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:24.510879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] PATH_INFO: `/v3/projects/30f30138096346329ab8d2a6c0db5e41/users/945a2dd8428e4edba616f1e3d7951921/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:24.513004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f30138096346329ab8d2a6c0db5e41, user_id=945a2dd8428e4edba616f1e3d7951921, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:24.515645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:24.526157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1163/2306] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:24 2022] PUT /identity/v3/projects/30f30138096346329ab8d2a6c0db5e41/users/945a2dd8428e4edba616f1e3d7951921/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:24.536256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8325d-f191-49c9-9f3b-e9c838b323a3 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:24.536590 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8325d-f191-49c9-9f3b-e9c838b323a3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:24.536884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8325d-f191-49c9-9f3b-e9c838b323a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:24.566766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4aa8325d-f191-49c9-9f3b-e9c838b323a3 None None] MFA Rules not processed for user `945a2dd8428e4edba616f1e3d7951921`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:24.568015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4aa8325d-f191-49c9-9f3b-e9c838b323a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:24.648329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1144/2307] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:24.659725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:24.661395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:24.718766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4f1f-26e5-4002-a9f6-a9764c358b82 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:24.722328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c3a91-3653-4bf2-8751-7aaceec52dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:24.722967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c3a91-3653-4bf2-8751-7aaceec52dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:24.723258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c3a91-3653-4bf2-8751-7aaceec52dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:24.723544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c3a91-3653-4bf2-8751-7aaceec52dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:24.776307 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:24.785213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c3a91-3653-4bf2-8751-7aaceec52dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:24.786693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c3a91-3653-4bf2-8751-7aaceec52dc3 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:24.796111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1164/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:24 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:26.634523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8325d-f191-49c9-9f3b-e9c838b323a3 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:26.636667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4aa8325d-f191-49c9-9f3b-e9c838b323a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:26.648722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8325d-f191-49c9-9f3b-e9c838b323a3 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:26.652295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34180970-a651-4698-95db-2c5a1550ced2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:26.652972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34180970-a651-4698-95db-2c5a1550ced2 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:26.653263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34180970-a651-4698-95db-2c5a1550ced2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:26.653545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34180970-a651-4698-95db-2c5a1550ced2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:26.685177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:26.713490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34180970-a651-4698-95db-2c5a1550ced2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:26.714925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34180970-a651-4698-95db-2c5a1550ced2 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:26.723922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1145/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:28.200846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c3a91-3653-4bf2-8751-7aaceec52dc3 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:28.211455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c3a91-3653-4bf2-8751-7aaceec52dc3 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:28.216089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e01e28-3bf3-4256-82ed-c6f18f42f55e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:28.216791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01e28-3bf3-4256-82ed-c6f18f42f55e service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:28.217071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01e28-3bf3-4256-82ed-c6f18f42f55e service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:28.217350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01e28-3bf3-4256-82ed-c6f18f42f55e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:28.226950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01e28-3bf3-4256-82ed-c6f18f42f55e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:28.228378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01e28-3bf3-4256-82ed-c6f18f42f55e service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:28.238937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1165/2310] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:28.791156 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4630c0f-7555-4be0-90ac-cfa1c0c82be9 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:28.791673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4630c0f-7555-4be0-90ac-cfa1c0c82be9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:28.791778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4630c0f-7555-4be0-90ac-cfa1c0c82be9 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:28.793213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1146/2311] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:37:28.799227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef829-422e-41a4-acdf-39ed8b048335 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:28.799522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef829-422e-41a4-acdf-39ed8b048335 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:28.799803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef829-422e-41a4-acdf-39ed8b048335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:28.860935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a36ef829-422e-41a4-acdf-39ed8b048335 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:28.862213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a36ef829-422e-41a4-acdf-39ed8b048335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:28.945506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1166/2312] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:37:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:28.957863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4630c0f-7555-4be0-90ac-cfa1c0c82be9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:28.959545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f4630c0f-7555-4be0-90ac-cfa1c0c82be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:29.017480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4630c0f-7555-4be0-90ac-cfa1c0c82be9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:29.021131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7d9af-31d1-47a4-86b8-5e40e2829750 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c958b233374b4013a8bbe9efbed054a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:29.021780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d9af-31d1-47a4-86b8-5e40e2829750 service glance] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:29.022065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d9af-31d1-47a4-86b8-5e40e2829750 service glance] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:29.022367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d9af-31d1-47a4-86b8-5e40e2829750 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:29.032419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7d9af-31d1-47a4-86b8-5e40e2829750 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:29.033866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7d9af-31d1-47a4-86b8-5e40e2829750 service glance] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:29.044499 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1147/2313] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:29.274777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a9cd-a7d1-4fa5-a6f0-7b5bc29c45d9 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:29.275222 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a9cd-a7d1-4fa5-a6f0-7b5bc29c45d9 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:29.275375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a9cd-a7d1-4fa5-a6f0-7b5bc29c45d9 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:29.276862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1167/2314] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:37:29.281821 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b65e37-42c3-47c5-b826-0da53edd75a3 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:29.282127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b65e37-42c3-47c5-b826-0da53edd75a3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:29.282414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b65e37-42c3-47c5-b826-0da53edd75a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:29.301046 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c3b65e37-42c3-47c5-b826-0da53edd75a3 None None] MFA Rules not processed for user `96df1ee2cbc14bcc8a8a250fd8f349dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:29.310909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c3b65e37-42c3-47c5-b826-0da53edd75a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:29.352005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1148/2315] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:37:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:29.364322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a9cd-a7d1-4fa5-a6f0-7b5bc29c45d9 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:29.366091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9d81a9cd-a7d1-4fa5-a6f0-7b5bc29c45d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:29.380036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a9cd-a7d1-4fa5-a6f0-7b5bc29c45d9 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:29.384945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6c190-cf54-4ee7-9672-fe01f383d481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:29.385588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c190-cf54-4ee7-9672-fe01f383d481 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:29.385871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c190-cf54-4ee7-9672-fe01f383d481 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:29.386238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c190-cf54-4ee7-9672-fe01f383d481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:29.393946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6c190-cf54-4ee7-9672-fe01f383d481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:29.395400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6c190-cf54-4ee7-9672-fe01f383d481 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:29.403416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1168/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:29.496864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af959b38-710b-496d-b336-c2e4927d5db7 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:29.497227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af959b38-710b-496d-b336-c2e4927d5db7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:29.497579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af959b38-710b-496d-b336-c2e4927d5db7 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:29.499271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1149/2317] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:37:29.511378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0c81a-a364-45ef-b3f7-9a223d71d510 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:29.512130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0c81a-a364-45ef-b3f7-9a223d71d510 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:29.512130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0c81a-a364-45ef-b3f7-9a223d71d510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:29.534091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-48b0c81a-a364-45ef-b3f7-9a223d71d510 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:29.535352 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-48b0c81a-a364-45ef-b3f7-9a223d71d510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:29.547326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1169/2318] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:37:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:29.558151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af959b38-710b-496d-b336-c2e4927d5db7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:29.570646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af959b38-710b-496d-b336-c2e4927d5db7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:29.574513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52539d1-175a-4cd3-99a3-a44f131bff98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:29.578720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52539d1-175a-4cd3-99a3-a44f131bff98 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:29.578720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52539d1-175a-4cd3-99a3-a44f131bff98 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:29.578720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52539d1-175a-4cd3-99a3-a44f131bff98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:29.587593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52539d1-175a-4cd3-99a3-a44f131bff98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:29.589681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52539d1-175a-4cd3-99a3-a44f131bff98 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:29.607564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1150/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:32.898876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0c81a-a364-45ef-b3f7-9a223d71d510 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:32.972916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0c81a-a364-45ef-b3f7-9a223d71d510 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:32.981006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:32.981006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.981006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.981006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] PATH_INFO: `/v3/users/af854687c48d437ba86ab3c7057d2750` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.981421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=af854687c48d437ba86ab3c7057d2750)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:32.982200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:32.984699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:33.181336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af854687c48d437ba86ab3c7057d2750'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:33.194383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af854687c48d437ba86ab3c7057d2750'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:33.202259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af854687c48d437ba86ab3c7057d2750'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:33.203577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af854687c48d437ba86ab3c7057d2750'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:33.204265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af854687c48d437ba86ab3c7057d2750'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:33.302952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1170/2320] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:32 2022] DELETE /identity/v3/users/af854687c48d437ba86ab3c7057d2750 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:33.307601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0f697-7711-4a45-94ad-1d2b15262967 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:33.309388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0f697-7711-4a45-94ad-1d2b15262967 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:33.309388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0f697-7711-4a45-94ad-1d2b15262967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:33.327184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-22b0f697-7711-4a45-94ad-1d2b15262967 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:33.328321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-22b0f697-7711-4a45-94ad-1d2b15262967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:33.389444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1151/2321] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:33.399640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:33.401167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:33.462186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4f0a8-d6f5-497c-be74-b5ba47bae5cc admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:33.466293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d26c0d-a493-4d31-9799-9ee6eacfb103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:33.466940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d26c0d-a493-4d31-9799-9ee6eacfb103 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:33.467225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d26c0d-a493-4d31-9799-9ee6eacfb103 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:33.467500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d26c0d-a493-4d31-9799-9ee6eacfb103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:33.476649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d26c0d-a493-4d31-9799-9ee6eacfb103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:33.478062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d26c0d-a493-4d31-9799-9ee6eacfb103 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:33.489904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1171/2322] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:33.913621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0f697-7711-4a45-94ad-1d2b15262967 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:33.915481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-22b0f697-7711-4a45-94ad-1d2b15262967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:33.916654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810f6ab-107e-4be4-a2a8-2acdf8fdc483 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:33.917588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810f6ab-107e-4be4-a2a8-2acdf8fdc483 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:33.917998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810f6ab-107e-4be4-a2a8-2acdf8fdc483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:33.929987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0f697-7711-4a45-94ad-1d2b15262967 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:33.934725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:33.935746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:33.936034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:33.936476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] PATH_INFO: `/v3/projects/39dfe24befeb4c568c343835b11624a4` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:33.938070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39dfe24befeb4c568c343835b11624a4)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:33.940731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:33.943828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-3810f6ab-107e-4be4-a2a8-2acdf8fdc483 None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:33.946038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3810f6ab-107e-4be4-a2a8-2acdf8fdc483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:33.948307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:34.011214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1172/2323] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:37:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:37:34.033994 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810f6ab-107e-4be4-a2a8-2acdf8fdc483 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:34.035708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3810f6ab-107e-4be4-a2a8-2acdf8fdc483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:34.040815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39dfe24befeb4c568c343835b11624a4', 'request_id': 'req-1a8c3600-4613-4e82-8975-c65d6dcf9868'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.071844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:34.107790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39dfe24befeb4c568c343835b11624a4', 'request_id': 'req-1a8c3600-4613-4e82-8975-c65d6dcf9868'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.112514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810f6ab-107e-4be4-a2a8-2acdf8fdc483 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:34.114178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39dfe24befeb4c568c343835b11624a4', 'request_id': 'req-1a8c3600-4613-4e82-8975-c65d6dcf9868'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.125748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] The token cache is being invalidate because project 39dfe24befeb4c568c343835b11624a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:34.126113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.126874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.128511 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1152/2324] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:33 2022] DELETE /identity/v3/projects/39dfe24befeb4c568c343835b11624a4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:34.133427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:34.134902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:34.175036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:34.176036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:34.176493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:34.176953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:34.179232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:34.217951 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3600-4613-4e82-8975-c65d6dcf9868 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:34.222670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:34.223548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:34.223832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:34.224111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] PATH_INFO: `/v3/users/358b6b799ad748429cd5b093848c50e4` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:34.225374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=358b6b799ad748429cd5b093848c50e4)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:34.227347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:34.240829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:34.242780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:34.251012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1173/2325] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:37:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:37:34.515723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '358b6b799ad748429cd5b093848c50e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.533772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '358b6b799ad748429cd5b093848c50e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.536836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '358b6b799ad748429cd5b093848c50e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.538880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '358b6b799ad748429cd5b093848c50e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.600174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '358b6b799ad748429cd5b093848c50e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.633122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1153/2326] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:34 2022] DELETE /identity/v3/users/358b6b799ad748429cd5b093848c50e4 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:35.132093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:35.163814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6644e8f-5858-440f-98e9-90012b0a993e service swift] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:35.230354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:35.231800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:35.232086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:35.232365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] PATH_INFO: `/v3/projects/dd46c6dc4d414dcc9f2c6b12f8e2a115` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:35.233621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd46c6dc4d414dcc9f2c6b12f8e2a115)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:35.235551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:35.253251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:35.338130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd46c6dc4d414dcc9f2c6b12f8e2a115', 'request_id': 'req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.411569 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd46c6dc4d414dcc9f2c6b12f8e2a115', 'request_id': 'req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.419110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd46c6dc4d414dcc9f2c6b12f8e2a115', 'request_id': 'req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.431691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] The token cache is being invalidate because project dd46c6dc4d414dcc9f2c6b12f8e2a115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:35.431995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.432665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.434253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1174/2327] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:35 2022] DELETE /identity/v3/projects/dd46c6dc4d414dcc9f2c6b12f8e2a115 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:35.438500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:35.440136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:35.455294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9045a81-f6c3-48c0-88f2-1582c777bae5 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:35.464658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:35.531456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:35.532346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:35.532666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:35.535084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] PATH_INFO: `/v3/users/945a2dd8428e4edba616f1e3d7951921` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:35.535084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=945a2dd8428e4edba616f1e3d7951921)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:35.536090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:35.729031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945a2dd8428e4edba616f1e3d7951921'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.736992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945a2dd8428e4edba616f1e3d7951921'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.737665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945a2dd8428e4edba616f1e3d7951921'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.738274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945a2dd8428e4edba616f1e3d7951921'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.834077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e13f1c9c-d795-4815-801a-b55e1bbc84c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945a2dd8428e4edba616f1e3d7951921'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:35.854886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1154/2328] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:35 2022] DELETE /identity/v3/users/945a2dd8428e4edba616f1e3d7951921 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:36.308295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:36.310358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:36.341675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b20ff-67fa-4acc-bf7e-68dbdadd1bc9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:36.423166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:36.424049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:36.424335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:36.424648 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] PATH_INFO: `/v3/projects/30f30138096346329ab8d2a6c0db5e41` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:36.425871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f30138096346329ab8d2a6c0db5e41)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:36.427836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:36.533497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f30138096346329ab8d2a6c0db5e41', 'request_id': 'req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:36.577080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f30138096346329ab8d2a6c0db5e41', 'request_id': 'req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:36.583129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f30138096346329ab8d2a6c0db5e41', 'request_id': 'req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:36.594849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] The token cache is being invalidate because project 30f30138096346329ab8d2a6c0db5e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:36.595138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:36.595785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-501cd7dd-72d3-4a45-8b22-65510ba1f9ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:36.597390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1175/2329] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:36 2022] DELETE /identity/v3/projects/30f30138096346329ab8d2a6c0db5e41 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:36.602429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3099c8-d5b4-423a-848b-9ad2b3659052 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:36.602961 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3099c8-d5b4-423a-848b-9ad2b3659052 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:36.603055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3099c8-d5b4-423a-848b-9ad2b3659052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:36.622814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7d3099c8-d5b4-423a-848b-9ad2b3659052 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:36.624012 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7d3099c8-d5b4-423a-848b-9ad2b3659052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:36.692165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1155/2330] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:36.700051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20906-61a3-4bd9-9d2c-eeb04b90b5a0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:36.702154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20906-61a3-4bd9-9d2c-eeb04b90b5a0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:36.702154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20906-61a3-4bd9-9d2c-eeb04b90b5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:36.722281 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d8b20906-61a3-4bd9-9d2c-eeb04b90b5a0 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:36.723547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d8b20906-61a3-4bd9-9d2c-eeb04b90b5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:36.732596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1176/2331] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:36.736541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3099c8-d5b4-423a-848b-9ad2b3659052 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:36.746882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3099c8-d5b4-423a-848b-9ad2b3659052 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:36.751437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:36.752108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:36.752393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:36.752724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:36.753464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:37:36.753780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:36.755411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:36.768096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1156/2332] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:37:36.772358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b20906-61a3-4bd9-9d2c-eeb04b90b5a0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:36.781879 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b20906-61a3-4bd9-9d2c-eeb04b90b5a0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:36.786666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8611318-6649-45d7-a9d7-30ef1faebe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:36.787359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8611318-6649-45d7-a9d7-30ef1faebe34 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:36.787647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8611318-6649-45d7-a9d7-30ef1faebe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:36.787926 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8611318-6649-45d7-a9d7-30ef1faebe34 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:36.788653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8611318-6649-45d7-a9d7-30ef1faebe34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:36.790833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8611318-6649-45d7-a9d7-30ef1faebe34 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:36.904868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1177/2333] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:36 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:36.908992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:36.920236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479983e5-b331-4295-ac63-9b913757839d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:36.943935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:36.994914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87451274-57e7-4b7a-a714-0673a6d6e986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:36.995684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87451274-57e7-4b7a-a714-0673a6d6e986 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:36.996132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87451274-57e7-4b7a-a714-0673a6d6e986 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:36.996486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87451274-57e7-4b7a-a714-0673a6d6e986 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:36.997288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87451274-57e7-4b7a-a714-0673a6d6e986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:36.999294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87451274-57e7-4b7a-a714-0673a6d6e986 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.099409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1157/2334] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:36 2022] POST /identity/v3/users => generated 284 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:37.108380 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8611318-6649-45d7-a9d7-30ef1faebe34 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.115632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:37.119131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8611318-6649-45d7-a9d7-30ef1faebe34 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.124594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba3630-f820-474a-9274-a9c8e66c655b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.125341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba3630-f820-474a-9274-a9c8e66c655b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.125627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba3630-f820-474a-9274-a9c8e66c655b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.125980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba3630-f820-474a-9274-a9c8e66c655b admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.126718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba3630-f820-474a-9274-a9c8e66c655b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.128312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba3630-f820-474a-9274-a9c8e66c655b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.141041 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1178/2335] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:37.145884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87451274-57e7-4b7a-a714-0673a6d6e986 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.156927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87451274-57e7-4b7a-a714-0673a6d6e986 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.161813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade561f-fcfa-4256-88cd-a83d089778ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.162485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade561f-fcfa-4256-88cd-a83d089778ae admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.163034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade561f-fcfa-4256-88cd-a83d089778ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.166360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade561f-fcfa-4256-88cd-a83d089778ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.166360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade561f-fcfa-4256-88cd-a83d089778ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.166360 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade561f-fcfa-4256-88cd-a83d089778ae admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.180668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1158/2336] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:37.182111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba3630-f820-474a-9274-a9c8e66c655b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.192612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba3630-f820-474a-9274-a9c8e66c655b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.198070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.198982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.199688 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.200334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] PATH_INFO: `/v3/projects/03a03b2a18664b2a826416647f20aa40/users/e9ee4b54b3f44a1cb393a3230be80a2d/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.219262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a03b2a18664b2a826416647f20aa40, user_id=e9ee4b54b3f44a1cb393a3230be80a2d, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.222066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.291548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1179/2337] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:37 2022] PUT /identity/v3/projects/03a03b2a18664b2a826416647f20aa40/users/e9ee4b54b3f44a1cb393a3230be80a2d/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:37.307632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601b77-3901-49e0-ac93-7275eb9a992c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.307924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601b77-3901-49e0-ac93-7275eb9a992c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.308206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601b77-3901-49e0-ac93-7275eb9a992c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.339070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c0601b77-3901-49e0-ac93-7275eb9a992c None None] MFA Rules not processed for user `e9ee4b54b3f44a1cb393a3230be80a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:37.341005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c0601b77-3901-49e0-ac93-7275eb9a992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:37.423048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1159/2338] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:37:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:37.434977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.437426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:37.506114 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174e903-85d6-453d-b03f-48e5a57ea6a3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.511188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eae3f-fb1c-4114-a98b-0947f5cbcfc3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c958b233374b4013a8bbe9efbed054a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.512468 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eae3f-fb1c-4114-a98b-0947f5cbcfc3 service glance] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.512795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eae3f-fb1c-4114-a98b-0947f5cbcfc3 service glance] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.513078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eae3f-fb1c-4114-a98b-0947f5cbcfc3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.523154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eae3f-fb1c-4114-a98b-0947f5cbcfc3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.524972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eae3f-fb1c-4114-a98b-0947f5cbcfc3 service glance] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.536575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1180/2339] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:47.138724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601b77-3901-49e0-ac93-7275eb9a992c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:47.140692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c0601b77-3901-49e0-ac93-7275eb9a992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:47.230342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601b77-3901-49e0-ac93-7275eb9a992c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:47.235570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:47.236253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:47.236561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:47.236846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:47.238673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:47.250510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:47.252079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:47.256354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:47.262257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1160/2340] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:47.535902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:57.318835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:10.522676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291e3f-8f1d-47ff-8e4c-a8ecf7857b3e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:10.523178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291e3f-8f1d-47ff-8e4c-a8ecf7857b3e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:10.523283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291e3f-8f1d-47ff-8e4c-a8ecf7857b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:10.595980 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2c291e3f-8f1d-47ff-8e4c-a8ecf7857b3e None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:38:10.600989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2c291e3f-8f1d-47ff-8e4c-a8ecf7857b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:10.660329 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1181/2341] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jun 16 00:38:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:38:10.673061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:10.674459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:10.762016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb3c8c-eb6d-42aa-93f8-5d60e812ed74 service placement] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:10.766960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73644b06-f36c-4350-97ea-fecf5a43336d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:10.767593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73644b06-f36c-4350-97ea-fecf5a43336d service swift] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:10.767878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73644b06-f36c-4350-97ea-fecf5a43336d service swift] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:10.768152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73644b06-f36c-4350-97ea-fecf5a43336d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:10.776454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73644b06-f36c-4350-97ea-fecf5a43336d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:10.778044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73644b06-f36c-4350-97ea-fecf5a43336d service swift] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:10.786637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1161/2342] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:38:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:38:20.657421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=40 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:20.782679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=41 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:25.360071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c291e3f-8f1d-47ff-8e4c-a8ecf7857b3e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:25.427807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c291e3f-8f1d-47ff-8e4c-a8ecf7857b3e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:25.433437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:25.434318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:25.434598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:25.434874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] PATH_INFO: `/v3/users/e9ee4b54b3f44a1cb393a3230be80a2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:25.436056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ee4b54b3f44a1cb393a3230be80a2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:25.438038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:25.620311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ee4b54b3f44a1cb393a3230be80a2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:25.634153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ee4b54b3f44a1cb393a3230be80a2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:25.643130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ee4b54b3f44a1cb393a3230be80a2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:25.645586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ee4b54b3f44a1cb393a3230be80a2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:25.646292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ee4b54b3f44a1cb393a3230be80a2d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:25.719171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1182/2343] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:38:25 2022] DELETE /identity/v3/users/e9ee4b54b3f44a1cb393a3230be80a2d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:38:25.723575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244b931-a991-45e8-bbfc-c28bae5c91d9 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:25.724580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244b931-a991-45e8-bbfc-c28bae5c91d9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:25.734101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244b931-a991-45e8-bbfc-c28bae5c91d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:25.748675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4244b931-a991-45e8-bbfc-c28bae5c91d9 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:38:25.750788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4244b931-a991-45e8-bbfc-c28bae5c91d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:25.814453 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1162/2344] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:38:25.824286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:25.825878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:25.889426 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631854-d1e7-4c22-a6c0-23cbba0c478c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:25.893004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd122bc-6497-4bd7-bcc3-3260116e4bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:25.893660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd122bc-6497-4bd7-bcc3-3260116e4bfe service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:25.893942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd122bc-6497-4bd7-bcc3-3260116e4bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:25.894251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd122bc-6497-4bd7-bcc3-3260116e4bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:25.903898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd122bc-6497-4bd7-bcc3-3260116e4bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:25.905396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd122bc-6497-4bd7-bcc3-3260116e4bfe service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:25.917265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1183/2345] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:26.384303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244b931-a991-45e8-bbfc-c28bae5c91d9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:26.385973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4244b931-a991-45e8-bbfc-c28bae5c91d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:26.399196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244b931-a991-45e8-bbfc-c28bae5c91d9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:26.405954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:26.407127 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:26.408015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:26.408331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] PATH_INFO: `/v3/projects/03a03b2a18664b2a826416647f20aa40` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:26.409779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a03b2a18664b2a826416647f20aa40)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:26.411911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:26.532132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a03b2a18664b2a826416647f20aa40', 'request_id': 'req-7975b490-dc1a-4e08-8412-ea9d85dcf20c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:26.621446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a03b2a18664b2a826416647f20aa40', 'request_id': 'req-7975b490-dc1a-4e08-8412-ea9d85dcf20c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:26.627467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a03b2a18664b2a826416647f20aa40', 'request_id': 'req-7975b490-dc1a-4e08-8412-ea9d85dcf20c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:26.641541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] The token cache is being invalidate because project 03a03b2a18664b2a826416647f20aa40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:38:26.641830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:26.642484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-7975b490-dc1a-4e08-8412-ea9d85dcf20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:38:26.644464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1163/2346] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:38:26 2022] DELETE /identity/v3/projects/03a03b2a18664b2a826416647f20aa40 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:38:33.068824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fa15-a40a-4c62-a29f-aa1f71d0ebad None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:33.069293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fa15-a40a-4c62-a29f-aa1f71d0ebad None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:33.069410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fa15-a40a-4c62-a29f-aa1f71d0ebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:33.087442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1d75fa15-a40a-4c62-a29f-aa1f71d0ebad None None] MFA Rules not processed for user `052ae1db33014bb5915dc228f1b76d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:38:33.088733 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1d75fa15-a40a-4c62-a29f-aa1f71d0ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:33.105328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:33.142420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1184/2347] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:38:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:38:34.956813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f9524-f2f7-4a7b-a7c3-861aa6efcce2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:34.957367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f9524-f2f7-4a7b-a7c3-861aa6efcce2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:34.957843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f9524-f2f7-4a7b-a7c3-861aa6efcce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:34.983665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8b0f9524-f2f7-4a7b-a7c3-861aa6efcce2 None None] MFA Rules not processed for user `6294a11cbb2a475da4864ca15b7959d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:38:34.984953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8b0f9524-f2f7-4a7b-a7c3-861aa6efcce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:35.064120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1164/2348] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:38:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:38:43.175381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=15 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:45.056346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:39:09.426285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d859b2-5cee-424a-bc92-399c1ac13258 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:09.426800 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d859b2-5cee-424a-bc92-399c1ac13258 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:09.426957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d859b2-5cee-424a-bc92-399c1ac13258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:09.454988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f0d859b2-5cee-424a-bc92-399c1ac13258 None None] MFA Rules not processed for user `c9dcfb252f0d44a280b3448094883e13`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:39:09.456312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f0d859b2-5cee-424a-bc92-399c1ac13258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:09.518967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1185/2349] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:39:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:39:13.738950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806951c1-0cea-4d17-8505-f7658b8ac17f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:13.739510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806951c1-0cea-4d17-8505-f7658b8ac17f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:13.739620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806951c1-0cea-4d17-8505-f7658b8ac17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:13.757766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-806951c1-0cea-4d17-8505-f7658b8ac17f None None] MFA Rules not processed for user `31973e5a04c14cfc90ab377eac5f4194`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:39:13.758987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-806951c1-0cea-4d17-8505-f7658b8ac17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:13.812788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1165/2350] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:39:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:39:19.507060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=25 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:39:23.806261 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=25 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:39:34.977774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d859b2-5cee-424a-bc92-399c1ac13258 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:34.979351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f0d859b2-5cee-424a-bc92-399c1ac13258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:35.056134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d859b2-5cee-424a-bc92-399c1ac13258 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:35.060319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4287a5-24bd-4973-b34a-e654aca79e4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:35.060997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4287a5-24bd-4973-b34a-e654aca79e4a service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:35.061279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4287a5-24bd-4973-b34a-e654aca79e4a service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:35.061557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4287a5-24bd-4973-b34a-e654aca79e4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:35.063322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2f4287a5-24bd-4973-b34a-e654aca79e4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:35.138036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4287a5-24bd-4973-b34a-e654aca79e4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:35.140029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4287a5-24bd-4973-b34a-e654aca79e4a service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:35.156584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1186/2351] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:39:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:39:35.324120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806951c1-0cea-4d17-8505-f7658b8ac17f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:35.333985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806951c1-0cea-4d17-8505-f7658b8ac17f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:35.337693 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:35.339690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:35.339690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:35.339690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:35.340681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:35.353814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:35.355206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:35.366575 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1166/2352] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:39:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:39:45.151539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=63 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:39:45.361760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=6 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:39:45.823810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f447e-dfbf-4b26-90ed-fb65de506df0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:45.823810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f447e-dfbf-4b26-90ed-fb65de506df0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:45.823810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f447e-dfbf-4b26-90ed-fb65de506df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:45.842313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-223f447e-dfbf-4b26-90ed-fb65de506df0 None None] MFA Rules not processed for user `052ae1db33014bb5915dc228f1b76d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:39:45.843531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-223f447e-dfbf-4b26-90ed-fb65de506df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:45.850618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1187/2353] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:39:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:39:45.979721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:45.981218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:46.050559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3703-5eff-4676-b530-6ed11851ec9e service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:46.055338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:46.056233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:46.056515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:46.056962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] PATH_INFO: `/v3/users/6294a11cbb2a475da4864ca15b7959d5` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:46.058199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6294a11cbb2a475da4864ca15b7959d5)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:46.060703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:46.206982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6294a11cbb2a475da4864ca15b7959d5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.214048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6294a11cbb2a475da4864ca15b7959d5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.214765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6294a11cbb2a475da4864ca15b7959d5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.215378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6294a11cbb2a475da4864ca15b7959d5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.266389 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6294a11cbb2a475da4864ca15b7959d5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.284976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1167/2354] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:39:45 2022] DELETE /identity/v3/users/6294a11cbb2a475da4864ca15b7959d5 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:39:46.573748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f447e-dfbf-4b26-90ed-fb65de506df0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:46.582614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:46.586339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:46.597038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f447e-dfbf-4b26-90ed-fb65de506df0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:46.664107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:46.665357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:46.665867 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:46.666146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] PATH_INFO: `/v3/projects/213787a3040e45efb2c14f5dd9dcf509` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:46.667580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213787a3040e45efb2c14f5dd9dcf509)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:46.670154 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:46.699531 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae09ba-7a06-4343-b97b-f9f80a67151a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:46.705327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c9b88-f3c0-4bdf-941c-160e72f684c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:46.706024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9b88-f3c0-4bdf-941c-160e72f684c9 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:46.706254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9b88-f3c0-4bdf-941c-160e72f684c9 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:46.706532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9b88-f3c0-4bdf-941c-160e72f684c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:46.708432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9d2c9b88-f3c0-4bdf-941c-160e72f684c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:46.723624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:39:46.769422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9b88-f3c0-4bdf-941c-160e72f684c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:46.771005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9b88-f3c0-4bdf-941c-160e72f684c9 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:46.780607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1168/2355] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:39:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:39:46.799876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213787a3040e45efb2c14f5dd9dcf509', 'request_id': 'req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.878181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213787a3040e45efb2c14f5dd9dcf509', 'request_id': 'req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.884979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213787a3040e45efb2c14f5dd9dcf509', 'request_id': 'req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.896440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] The token cache is being invalidate because project 213787a3040e45efb2c14f5dd9dcf509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:39:46.896767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.897542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0dd78ce3-2cca-4453-a2b5-90f6a74e7185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:39:46.899118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1188/2356] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:39:46 2022] DELETE /identity/v3/projects/213787a3040e45efb2c14f5dd9dcf509 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:39:46.904620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9fc0-1f0e-4f1e-b9bc-6030b317ea60 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:46.904917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9fc0-1f0e-4f1e-b9bc-6030b317ea60 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:46.905199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9fc0-1f0e-4f1e-b9bc-6030b317ea60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:46.923909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c7aa9fc0-1f0e-4f1e-b9bc-6030b317ea60 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:39:46.925174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c7aa9fc0-1f0e-4f1e-b9bc-6030b317ea60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:46.985344 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1169/2357] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:39:46.992797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a39b2-a2d2-4b4e-a5ee-0e4b12b7f049 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:46.993093 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a39b2-a2d2-4b4e-a5ee-0e4b12b7f049 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:46.993378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a39b2-a2d2-4b4e-a5ee-0e4b12b7f049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.012023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-993a39b2-a2d2-4b4e-a5ee-0e4b12b7f049 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:39:47.020201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-993a39b2-a2d2-4b4e-a5ee-0e4b12b7f049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:47.021668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1189/2358] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:39:47.026070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9fc0-1f0e-4f1e-b9bc-6030b317ea60 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:47.036604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9fc0-1f0e-4f1e-b9bc-6030b317ea60 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:47.041117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:47.041787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:47.042072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:47.042351 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.043075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:39:47.043382 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:47.045066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:47.056784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1170/2359] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:39:47.060906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a39b2-a2d2-4b4e-a5ee-0e4b12b7f049 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:47.070414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a39b2-a2d2-4b4e-a5ee-0e4b12b7f049 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:47.075175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec350-5f12-4fcf-aa0c-f2025d1dde14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:47.076502 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec350-5f12-4fcf-aa0c-f2025d1dde14 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:47.076651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec350-5f12-4fcf-aa0c-f2025d1dde14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:47.076889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec350-5f12-4fcf-aa0c-f2025d1dde14 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.077578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bec350-5f12-4fcf-aa0c-f2025d1dde14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:47.079544 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bec350-5f12-4fcf-aa0c-f2025d1dde14 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:47.159600 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1190/2360] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:39:47 2022] POST /identity/v3/projects => generated 391 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:39:47.163662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:47.173862 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e8908-4194-48bf-977a-c7564671192f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:47.236805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ea9dd-f7d0-4acf-a9d4-11d06d030caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:47.237480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ea9dd-f7d0-4acf-a9d4-11d06d030caa admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:47.237770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ea9dd-f7d0-4acf-a9d4-11d06d030caa admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:47.238051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ea9dd-f7d0-4acf-a9d4-11d06d030caa admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.238752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ea9dd-f7d0-4acf-a9d4-11d06d030caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:47.240692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ea9dd-f7d0-4acf-a9d4-11d06d030caa admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:47.312413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1171/2361] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:39:47 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:39:47.316866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec350-5f12-4fcf-aa0c-f2025d1dde14 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:47.331121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec350-5f12-4fcf-aa0c-f2025d1dde14 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:47.335905 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ac002-8dd9-4d54-9500-cbf823f6aeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:47.336694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ac002-8dd9-4d54-9500-cbf823f6aeee admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:47.336983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ac002-8dd9-4d54-9500-cbf823f6aeee admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:47.337266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ac002-8dd9-4d54-9500-cbf823f6aeee admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.338029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ac002-8dd9-4d54-9500-cbf823f6aeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:47.339674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ac002-8dd9-4d54-9500-cbf823f6aeee admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:47.351487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1191/2362] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:39:47.354835 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ea9dd-f7d0-4acf-a9d4-11d06d030caa admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:47.365129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ea9dd-f7d0-4acf-a9d4-11d06d030caa admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:47.369764 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58ee6c-4bad-47b1-98a4-c793a5b90d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:47.370418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ee6c-4bad-47b1-98a4-c793a5b90d47 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:47.370706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ee6c-4bad-47b1-98a4-c793a5b90d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:47.370988 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ee6c-4bad-47b1-98a4-c793a5b90d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.371744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58ee6c-4bad-47b1-98a4-c793a5b90d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:47.373870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58ee6c-4bad-47b1-98a4-c793a5b90d47 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:47.384582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1172/2363] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:39:47.389198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ac002-8dd9-4d54-9500-cbf823f6aeee admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:47.399335 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ac002-8dd9-4d54-9500-cbf823f6aeee admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:47.404997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:47.405884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:47.406168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:47.406446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] PATH_INFO: `/v3/projects/317fdd1c7670491a9abe0398e63ae482/users/86ba71e145a14cc2b8be2feb2c5c0da1/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.422165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=317fdd1c7670491a9abe0398e63ae482, user_id=86ba71e145a14cc2b8be2feb2c5c0da1, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:47.424951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:47.565294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1192/2364] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:39:47 2022] PUT /identity/v3/projects/317fdd1c7670491a9abe0398e63ae482/users/86ba71e145a14cc2b8be2feb2c5c0da1/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:39:47.576362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac8bec-c763-4c3e-8dbe-17f816ee49df None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:47.576788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac8bec-c763-4c3e-8dbe-17f816ee49df None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:47.577097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac8bec-c763-4c3e-8dbe-17f816ee49df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.603878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-91ac8bec-c763-4c3e-8dbe-17f816ee49df None None] MFA Rules not processed for user `86ba71e145a14cc2b8be2feb2c5c0da1`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:39:47.605151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-91ac8bec-c763-4c3e-8dbe-17f816ee49df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:47.614175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:39:47.669064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1173/2365] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:39:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:39:47.683675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:47.686108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:47.709347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:39:47.744150 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ba9d8-0376-427b-a355-c3885d08e735 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:47.748608 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:47.750280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:47.750280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:47.750280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:47.758172 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:47.759646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:47.768757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1193/2366] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:39:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:39:55.933963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac8bec-c763-4c3e-8dbe-17f816ee49df None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:39:55.935580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-91ac8bec-c763-4c3e-8dbe-17f816ee49df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:56.009444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac8bec-c763-4c3e-8dbe-17f816ee49df None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:39:56.014876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:39:56.015517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:39:56.015799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:39:56.016075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:39:56.017965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:39:56.077053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:39:56.078640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:39:56.087985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1174/2367] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:39:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:39:58.540760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=17 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:04.899542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:04.901182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:04.958638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f9dab-67e4-43fc-a299-3ec2c6060a48 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:04.962302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9cd6-baaf-4e16-99af-0b8b617c32f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:04.962949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9cd6-baaf-4e16-99af-0b8b617c32f6 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:04.963236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9cd6-baaf-4e16-99af-0b8b617c32f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:04.963609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9cd6-baaf-4e16-99af-0b8b617c32f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:04.965303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-77ce9cd6-baaf-4e16-99af-0b8b617c32f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:04.979198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9cd6-baaf-4e16-99af-0b8b617c32f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:04.979198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9cd6-baaf-4e16-99af-0b8b617c32f6 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:04.985572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1194/2368] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:40:06.083501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=78 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:08.301684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:08.367010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa718f4-f915-4909-8f2b-ad1f4630af2b service nova] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:08.372104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:08.372996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:08.373277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:08.373551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] PATH_INFO: `/v3/users/86ba71e145a14cc2b8be2feb2c5c0da1` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:08.375539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ba71e145a14cc2b8be2feb2c5c0da1)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:08.377469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:08.537244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ba71e145a14cc2b8be2feb2c5c0da1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:08.543917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ba71e145a14cc2b8be2feb2c5c0da1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:08.544614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ba71e145a14cc2b8be2feb2c5c0da1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:08.545211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ba71e145a14cc2b8be2feb2c5c0da1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:08.614942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ba71e145a14cc2b8be2feb2c5c0da1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:08.634854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1175/2369] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:08 2022] DELETE /identity/v3/users/86ba71e145a14cc2b8be2feb2c5c0da1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:08.639429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa172fb-c873-4183-b68b-edb0f78cd5f2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:08.639916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa172fb-c873-4183-b68b-edb0f78cd5f2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:08.640050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa172fb-c873-4183-b68b-edb0f78cd5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:08.658201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-1fa172fb-c873-4183-b68b-edb0f78cd5f2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:08.659402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1fa172fb-c873-4183-b68b-edb0f78cd5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:08.718851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1195/2370] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:08.728205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:08.729729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:08.786849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14235f7e-4842-4e2b-8017-7c0c879b3bf8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:08.791663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:08.791663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:08.791663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:08.791974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:08.801734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:08.803237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:08.813618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1176/2371] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:09.425247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa172fb-c873-4183-b68b-edb0f78cd5f2 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:09.426666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1fa172fb-c873-4183-b68b-edb0f78cd5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:09.475124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:09.476598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:09.501208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d73d1f-97e0-4a6f-98ab-23eb98585271 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:09.518083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa172fb-c873-4183-b68b-edb0f78cd5f2 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:09.520478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:09.520478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:09.520478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:09.520478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] PATH_INFO: `/v3/projects/317fdd1c7670491a9abe0398e63ae482` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:09.525451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=317fdd1c7670491a9abe0398e63ae482)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:09.526194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e68cc-5ee9-44e4-bac2-62cc3266b365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:09.527491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e68cc-5ee9-44e4-bac2-62cc3266b365 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:09.527640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:09.527728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e68cc-5ee9-44e4-bac2-62cc3266b365 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:09.527806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e68cc-5ee9-44e4-bac2-62cc3266b365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:09.530381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1d9e68cc-5ee9-44e4-bac2-62cc3266b365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:09.545878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:09.549832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e68cc-5ee9-44e4-bac2-62cc3266b365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:09.552188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e68cc-5ee9-44e4-bac2-62cc3266b365 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:09.565133 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1196/2372] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:09.667731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317fdd1c7670491a9abe0398e63ae482', 'request_id': 'req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:09.763499 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317fdd1c7670491a9abe0398e63ae482', 'request_id': 'req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:09.771966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317fdd1c7670491a9abe0398e63ae482', 'request_id': 'req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:09.786013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] The token cache is being invalidate because project 317fdd1c7670491a9abe0398e63ae482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:09.786305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:09.787085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-b9c07cf0-351b-4d9c-be15-d5fe4f2b68d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:09.792044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1177/2373] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:09 2022] DELETE /identity/v3/projects/317fdd1c7670491a9abe0398e63ae482 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:09.793975 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819589d-70d1-4be3-91dc-6c8910b20a97 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:09.794447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819589d-70d1-4be3-91dc-6c8910b20a97 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:09.794552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819589d-70d1-4be3-91dc-6c8910b20a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:09.817451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2819589d-70d1-4be3-91dc-6c8910b20a97 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:09.818724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2819589d-70d1-4be3-91dc-6c8910b20a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:09.847288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:09.892924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1197/2374] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:09.895366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54ffd1-ea76-4ca0-8ce8-9164ffb84f4a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:09.895366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54ffd1-ea76-4ca0-8ce8-9164ffb84f4a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:09.895615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54ffd1-ea76-4ca0-8ce8-9164ffb84f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:09.915681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8f54ffd1-ea76-4ca0-8ce8-9164ffb84f4a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:09.916960 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8f54ffd1-ea76-4ca0-8ce8-9164ffb84f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:09.926126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1178/2375] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:09.931825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819589d-70d1-4be3-91dc-6c8910b20a97 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:09.946029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819589d-70d1-4be3-91dc-6c8910b20a97 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:09.951495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:09.955218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:09.955218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:09.955218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:09.955218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:40:09.955218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:09.956128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:09.971824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1198/2376] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:40:09.973104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54ffd1-ea76-4ca0-8ce8-9164ffb84f4a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:09.982737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54ffd1-ea76-4ca0-8ce8-9164ffb84f4a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:09.989131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4d39-572b-4f3c-a6eb-d7f3e725d37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:09.989131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4d39-572b-4f3c-a6eb-d7f3e725d37b admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:09.989364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4d39-572b-4f3c-a6eb-d7f3e725d37b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:09.989465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4d39-572b-4f3c-a6eb-d7f3e725d37b admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:09.990211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a4d39-572b-4f3c-a6eb-d7f3e725d37b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:09.992092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a4d39-572b-4f3c-a6eb-d7f3e725d37b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:10.083377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1179/2377] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:09 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:10.085848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:10.097941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6e80b-4bf6-4bd6-8140-4a0cc1a40c66 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:10.163295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79d9d8-60c5-4cbc-b519-ed27dd2a6a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:10.163979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79d9d8-60c5-4cbc-b519-ed27dd2a6a92 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:10.164266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79d9d8-60c5-4cbc-b519-ed27dd2a6a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:10.165334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79d9d8-60c5-4cbc-b519-ed27dd2a6a92 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:10.165455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79d9d8-60c5-4cbc-b519-ed27dd2a6a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:10.167478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79d9d8-60c5-4cbc-b519-ed27dd2a6a92 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:10.244379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1199/2378] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:40:10 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:40:10.254800 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4d39-572b-4f3c-a6eb-d7f3e725d37b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:10.267005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4d39-572b-4f3c-a6eb-d7f3e725d37b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:10.272833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd98a-efad-49ce-85c4-d97f01d1b9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:10.275906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd98a-efad-49ce-85c4-d97f01d1b9cf admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:10.275906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd98a-efad-49ce-85c4-d97f01d1b9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:10.275906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd98a-efad-49ce-85c4-d97f01d1b9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:10.275906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefd98a-efad-49ce-85c4-d97f01d1b9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:10.277113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefd98a-efad-49ce-85c4-d97f01d1b9cf admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:10.293694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1180/2379] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:10.297810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79d9d8-60c5-4cbc-b519-ed27dd2a6a92 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:10.308469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79d9d8-60c5-4cbc-b519-ed27dd2a6a92 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:10.314071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82048202-1917-4ad6-8a7e-f6f9dbd33b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:10.314749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82048202-1917-4ad6-8a7e-f6f9dbd33b23 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:10.315043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82048202-1917-4ad6-8a7e-f6f9dbd33b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:10.315332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82048202-1917-4ad6-8a7e-f6f9dbd33b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:10.316073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82048202-1917-4ad6-8a7e-f6f9dbd33b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:10.317734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82048202-1917-4ad6-8a7e-f6f9dbd33b23 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:10.331668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1200/2380] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:10.332887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd98a-efad-49ce-85c4-d97f01d1b9cf admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:10.343192 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd98a-efad-49ce-85c4-d97f01d1b9cf admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:10.348568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:10.349494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:10.349783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:10.350264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] PATH_INFO: `/v3/projects/ad47d922a30a4c55a781c52c5bd5236f/users/f2aa1b8204b1494589e096ad775ce4d9/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:10.366760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad47d922a30a4c55a781c52c5bd5236f, user_id=f2aa1b8204b1494589e096ad775ce4d9, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:10.369753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:10.461598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1181/2381] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:10 2022] PUT /identity/v3/projects/ad47d922a30a4c55a781c52c5bd5236f/users/f2aa1b8204b1494589e096ad775ce4d9/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:10.466680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f157cb-5965-4bfb-bb86-9fc0163e9121 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:10.467020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f157cb-5965-4bfb-bb86-9fc0163e9121 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:10.467304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f157cb-5965-4bfb-bb86-9fc0163e9121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:10.485782 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d9f157cb-5965-4bfb-bb86-9fc0163e9121 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:10.487070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d9f157cb-5965-4bfb-bb86-9fc0163e9121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:10.551344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1201/2382] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:10.561129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:10.562935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:10.623383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21851-92cc-4cc5-a00e-3ae7dcc3020a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:10.627319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:10.627970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:10.628261 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:10.628565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:10.637942 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:10.639372 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:10.649225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1182/2383] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:14.287894 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f157cb-5965-4bfb-bb86-9fc0163e9121 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:14.289243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d9f157cb-5965-4bfb-bb86-9fc0163e9121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:14.335277 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:14.376465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f157cb-5965-4bfb-bb86-9fc0163e9121 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:14.383665 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cace92-8552-460b-8dcc-f1b38883c76e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:14.384567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cace92-8552-460b-8dcc-f1b38883c76e service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:14.384757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cace92-8552-460b-8dcc-f1b38883c76e service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:14.385143 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cace92-8552-460b-8dcc-f1b38883c76e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:14.387045 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f2cace92-8552-460b-8dcc-f1b38883c76e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:14.451149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cace92-8552-460b-8dcc-f1b38883c76e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:14.452943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cace92-8552-460b-8dcc-f1b38883c76e service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:14.463804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1202/2384] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:16.161537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:16.163508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:16.187667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a911c9-b4a9-4733-8c89-62e5226d795f service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:16.197524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:16.198429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:16.199040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:16.199647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:16.202216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:16.236992 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:16.238628 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:16.251518 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1183/2385] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:16.483816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ed34f-d89f-40cc-a4ac-8f44056a70db None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:16.488978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ed34f-d89f-40cc-a4ac-8f44056a70db None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:16.488978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ed34f-d89f-40cc-a4ac-8f44056a70db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:16.522434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-797ed34f-d89f-40cc-a4ac-8f44056a70db None None] MFA Rules not processed for user `f2aa1b8204b1494589e096ad775ce4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:16.523605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-797ed34f-d89f-40cc-a4ac-8f44056a70db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:16.617893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1203/2386] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:16.622565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:16.633150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da615a6-c0fe-4bf2-bdb4-34a80c526cae service nova] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:16.637735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f0aa-b799-40e5-8c02-b332ab2eaf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:16.638385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f0aa-b799-40e5-8c02-b332ab2eaf64 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:16.638674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f0aa-b799-40e5-8c02-b332ab2eaf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:16.638958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f0aa-b799-40e5-8c02-b332ab2eaf64 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:16.639653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63f0aa-b799-40e5-8c02-b332ab2eaf64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:16.641696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63f0aa-b799-40e5-8c02-b332ab2eaf64 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:16.743559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1184/2387] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:16 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:40:16.746996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ed34f-d89f-40cc-a4ac-8f44056a70db None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:16.758319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ed34f-d89f-40cc-a4ac-8f44056a70db None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:16.815618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2cb9c-71fe-4555-9903-8446eced7dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:16.816223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2cb9c-71fe-4555-9903-8446eced7dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:16.816504 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2cb9c-71fe-4555-9903-8446eced7dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:16.816812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2cb9c-71fe-4555-9903-8446eced7dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:16.817505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2cb9c-71fe-4555-9903-8446eced7dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:16.819755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2cb9c-71fe-4555-9903-8446eced7dc4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:16.827311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:16.936939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1204/2388] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:40:16 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:16.944598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f0aa-b799-40e5-8c02-b332ab2eaf64 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:16.958901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f0aa-b799-40e5-8c02-b332ab2eaf64 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:16.963945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a69c5-5af5-46c6-be39-6f37c0de8312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:16.964651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a69c5-5af5-46c6-be39-6f37c0de8312 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:16.964983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a69c5-5af5-46c6-be39-6f37c0de8312 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:16.965273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a69c5-5af5-46c6-be39-6f37c0de8312 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:16.966593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a69c5-5af5-46c6-be39-6f37c0de8312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:16.967603 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a69c5-5af5-46c6-be39-6f37c0de8312 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:16.969944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:16.983288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1185/2389] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:16.984989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2cb9c-71fe-4555-9903-8446eced7dc4 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:16.995556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2cb9c-71fe-4555-9903-8446eced7dc4 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:17.000872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b36f9b-d879-4083-a3ea-fdc516d75ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:17.001775 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36f9b-d879-4083-a3ea-fdc516d75ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:17.003375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36f9b-d879-4083-a3ea-fdc516d75ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:17.003375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36f9b-d879-4083-a3ea-fdc516d75ad2 admin admin] PATH_INFO: `/v3/projects/d9eec0a2c24d4981997c685919bebeaa/users/4bf1fbaebebf4d5cb4c24a69dd123261/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:17.018260 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36f9b-d879-4083-a3ea-fdc516d75ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9eec0a2c24d4981997c685919bebeaa, user_id=4bf1fbaebebf4d5cb4c24a69dd123261, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:17.021339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36f9b-d879-4083-a3ea-fdc516d75ad2 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:17.112922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1205/2390] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:16 2022] PUT /identity/v3/projects/d9eec0a2c24d4981997c685919bebeaa/users/4bf1fbaebebf4d5cb4c24a69dd123261/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:17.117194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a69c5-5af5-46c6-be39-6f37c0de8312 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:17.128576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a69c5-5af5-46c6-be39-6f37c0de8312 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:17.185823 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f86cc0-bc86-4ea9-b3b5-453ea702e54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:17.186490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f86cc0-bc86-4ea9-b3b5-453ea702e54d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:17.186778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f86cc0-bc86-4ea9-b3b5-453ea702e54d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:17.187072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f86cc0-bc86-4ea9-b3b5-453ea702e54d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:17.188383 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f86cc0-bc86-4ea9-b3b5-453ea702e54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:17.190001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f86cc0-bc86-4ea9-b3b5-453ea702e54d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:17.199927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1186/2391] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:17.204803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b36f9b-d879-4083-a3ea-fdc516d75ad2 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:17.219500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b36f9b-d879-4083-a3ea-fdc516d75ad2 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:17.224659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:17.225860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:17.226145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:17.226421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] PATH_INFO: `/v3/projects/d9eec0a2c24d4981997c685919bebeaa/users/4bf1fbaebebf4d5cb4c24a69dd123261/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:17.228761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9eec0a2c24d4981997c685919bebeaa, user_id=4bf1fbaebebf4d5cb4c24a69dd123261, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:17.234851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:17.245816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1206/2392] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:17 2022] PUT /identity/v3/projects/d9eec0a2c24d4981997c685919bebeaa/users/4bf1fbaebebf4d5cb4c24a69dd123261/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:23.474311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c566a2-d418-435b-9b61-dcd745817aa5 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:23.475150 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c566a2-d418-435b-9b61-dcd745817aa5 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:23.475576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c566a2-d418-435b-9b61-dcd745817aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:23.505206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a1c566a2-d418-435b-9b61-dcd745817aa5 None None] MFA Rules not processed for user `4bf1fbaebebf4d5cb4c24a69dd123261`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:23.506559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a1c566a2-d418-435b-9b61-dcd745817aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:23.585745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1187/2393] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:23.597886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:23.599779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:23.733031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3adc6-2a21-4cd3-9133-45760ce80bf3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:23.737496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c958b233374b4013a8bbe9efbed054a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:23.738213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:23.738445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:23.738819 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:23.797492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:23.799175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:23.809599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1207/2394] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:24.033305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3bb8-0a6c-421c-8bde-4ee8cd68f1f0 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:24.033602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3bb8-0a6c-421c-8bde-4ee8cd68f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:24.033888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3bb8-0a6c-421c-8bde-4ee8cd68f1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:24.054489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c13a3bb8-0a6c-421c-8bde-4ee8cd68f1f0 None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:24.055715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c13a3bb8-0a6c-421c-8bde-4ee8cd68f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:24.114065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1188/2395] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:40:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:24.242483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:24.243888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:24.296089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:24.316772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d85a1-7c14-42c8-ba35-0e567a0026c2 service glance] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:24.322582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:24.323318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:24.323897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:24.324007 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:24.333348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:24.335013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:24.344364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1208/2396] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:40:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:40:24.363481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a3bb8-0a6c-421c-8bde-4ee8cd68f1f0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:24.364422 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c13a3bb8-0a6c-421c-8bde-4ee8cd68f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:24.399402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:24.438614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a3bb8-0a6c-421c-8bde-4ee8cd68f1f0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:24.442668 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:24.443693 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:24.444219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:24.444846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:24.447151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:24.546194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:24.547698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:24.559975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1189/2397] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:29.931193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:29.933348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:30.023384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d60ddc-616f-40e9-8178-6a1db02fe0ef service swift] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:30.028981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:30.029881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:30.030237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:30.031278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:30.034437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:30.059109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:30.060956 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:30.084838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1209/2398] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:30.164896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:30.167881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:30.185488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1ea90-6d12-466d-96eb-8889227f47c3 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:30.191020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:30.192148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:30.192785 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:30.193147 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] PATH_INFO: `/v3/users/c9dcfb252f0d44a280b3448094883e13` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:30.194729 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9dcfb252f0d44a280b3448094883e13)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:30.196774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:30.363396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9dcfb252f0d44a280b3448094883e13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:30.371001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9dcfb252f0d44a280b3448094883e13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:30.371732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9dcfb252f0d44a280b3448094883e13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:30.372342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9dcfb252f0d44a280b3448094883e13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:30.436793 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2d5e1d9-033d-4b8d-b849-5b90097bb0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9dcfb252f0d44a280b3448094883e13'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:30.470128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1190/2399] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:30 2022] DELETE /identity/v3/users/c9dcfb252f0d44a280b3448094883e13 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:30.742540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:30.759072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f791d6-fbc9-4697-bd4f-9fab4b2d16b2 service placement] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:30.842526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:30.843565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:30.843977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:30.844263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] PATH_INFO: `/v3/projects/a4e7be42e39248e4b9f5c89e96a6b82e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:30.846392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e7be42e39248e4b9f5c89e96a6b82e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:30.848308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:30.991090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e7be42e39248e4b9f5c89e96a6b82e', 'request_id': 'req-aec8e02f-c8c1-49a2-8792-769e8961bc5f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.079289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e7be42e39248e4b9f5c89e96a6b82e', 'request_id': 'req-aec8e02f-c8c1-49a2-8792-769e8961bc5f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.086999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e7be42e39248e4b9f5c89e96a6b82e', 'request_id': 'req-aec8e02f-c8c1-49a2-8792-769e8961bc5f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.094963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:31.099999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] The token cache is being invalidate because project a4e7be42e39248e4b9f5c89e96a6b82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:31.100295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.101122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-aec8e02f-c8c1-49a2-8792-769e8961bc5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.104205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1210/2400] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:30 2022] DELETE /identity/v3/projects/a4e7be42e39248e4b9f5c89e96a6b82e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:40:31.108100 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21bcf-bc07-4b4c-825d-73af000d167a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.108570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21bcf-bc07-4b4c-825d-73af000d167a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.108977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21bcf-bc07-4b4c-825d-73af000d167a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.134244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-99f21bcf-bc07-4b4c-825d-73af000d167a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:31.135606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-99f21bcf-bc07-4b4c-825d-73af000d167a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:31.190477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:31.220724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1191/2401] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:40:31.224896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e237-2cdf-46d9-9a9a-c79b07886754 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.225220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e237-2cdf-46d9-9a9a-c79b07886754 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.225603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e237-2cdf-46d9-9a9a-c79b07886754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.247747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-9871e237-2cdf-46d9-9a9a-c79b07886754 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:31.247747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9871e237-2cdf-46d9-9a9a-c79b07886754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:31.260245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1211/2402] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:40:31.261620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21bcf-bc07-4b4c-825d-73af000d167a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:31.276209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21bcf-bc07-4b4c-825d-73af000d167a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:31.281523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:31.282288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.282585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.282867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.283632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:40:31.284015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:31.285730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:31.304692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1192/2403] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:40:31.305384 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871e237-2cdf-46d9-9a9a-c79b07886754 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:31.322003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871e237-2cdf-46d9-9a9a-c79b07886754 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:31.326964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7f8e1-5409-4ef5-8521-77d11751e374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:31.327676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7f8e1-5409-4ef5-8521-77d11751e374 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.328678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7f8e1-5409-4ef5-8521-77d11751e374 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.328678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7f8e1-5409-4ef5-8521-77d11751e374 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.329050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7f8e1-5409-4ef5-8521-77d11751e374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:31.331066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7f8e1-5409-4ef5-8521-77d11751e374 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:31.806350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1212/2404] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:31 2022] POST /identity/v3/projects => generated 379 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:31.810991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:31.821691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe9545-4735-47e2-ba1c-c410b7a90e37 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:31.879570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdaae1-6a07-4c6b-96af-db875be25cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:31.880193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdaae1-6a07-4c6b-96af-db875be25cdd admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.880480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdaae1-6a07-4c6b-96af-db875be25cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.880792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdaae1-6a07-4c6b-96af-db875be25cdd admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.881495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdaae1-6a07-4c6b-96af-db875be25cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:31.883439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdaae1-6a07-4c6b-96af-db875be25cdd admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:31.966436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1193/2405] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:40:31 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:31.971087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7f8e1-5409-4ef5-8521-77d11751e374 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:31.983101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7f8e1-5409-4ef5-8521-77d11751e374 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:31.988175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa5f3a-4f1c-4459-b787-df220cf5e427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:31.989123 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5f3a-4f1c-4459-b787-df220cf5e427 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.989555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5f3a-4f1c-4459-b787-df220cf5e427 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.990027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5f3a-4f1c-4459-b787-df220cf5e427 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.991156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa5f3a-4f1c-4459-b787-df220cf5e427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:31.992858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa5f3a-4f1c-4459-b787-df220cf5e427 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.008368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1213/2406] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:32.009732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdaae1-6a07-4c6b-96af-db875be25cdd admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.021305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdaae1-6a07-4c6b-96af-db875be25cdd admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.027336 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda67c20-0aa1-4e21-90ba-b6e6e29f3aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.028001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67c20-0aa1-4e21-90ba-b6e6e29f3aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.028283 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67c20-0aa1-4e21-90ba-b6e6e29f3aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.028611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67c20-0aa1-4e21-90ba-b6e6e29f3aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.029334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda67c20-0aa1-4e21-90ba-b6e6e29f3aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.031652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda67c20-0aa1-4e21-90ba-b6e6e29f3aa6 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.047503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1194/2407] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:40:32.049437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa5f3a-4f1c-4459-b787-df220cf5e427 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.060068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa5f3a-4f1c-4459-b787-df220cf5e427 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.066069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc470a-d412-44a0-af7e-29405a016b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.067172 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc470a-d412-44a0-af7e-29405a016b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.067710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc470a-d412-44a0-af7e-29405a016b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.067995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc470a-d412-44a0-af7e-29405a016b69 admin admin] PATH_INFO: `/v3/projects/a9cf1ac6d8114d37b7658673777008ce/users/c9487779b7c24cf4a90ee61ae9a6a0c7/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.087779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc470a-d412-44a0-af7e-29405a016b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9cf1ac6d8114d37b7658673777008ce, user_id=c9487779b7c24cf4a90ee61ae9a6a0c7, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.090625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc470a-d412-44a0-af7e-29405a016b69 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.158692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1214/2408] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:32 2022] PUT /identity/v3/projects/a9cf1ac6d8114d37b7658673777008ce/users/c9487779b7c24cf4a90ee61ae9a6a0c7/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:32.168881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f1bd-58fd-45df-9bd0-7b7986e2d7f2 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.172076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f1bd-58fd-45df-9bd0-7b7986e2d7f2 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.172076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f1bd-58fd-45df-9bd0-7b7986e2d7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.203064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ec38f1bd-58fd-45df-9bd0-7b7986e2d7f2 None None] MFA Rules not processed for user `c9487779b7c24cf4a90ee61ae9a6a0c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:32.204277 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ec38f1bd-58fd-45df-9bd0-7b7986e2d7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:32.281111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1195/2409] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:32.285531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc470a-d412-44a0-af7e-29405a016b69 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.360680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc470a-d412-44a0-af7e-29405a016b69 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.365282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d252c7-5b91-403f-bc16-8d6cd5197f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.365953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d252c7-5b91-403f-bc16-8d6cd5197f80 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.366243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d252c7-5b91-403f-bc16-8d6cd5197f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.366524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d252c7-5b91-403f-bc16-8d6cd5197f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.367259 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d252c7-5b91-403f-bc16-8d6cd5197f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.369276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d252c7-5b91-403f-bc16-8d6cd5197f80 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.510055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1215/2410] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:32 2022] POST /identity/v3/projects => generated 381 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:32.514327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38f1bd-58fd-45df-9bd0-7b7986e2d7f2 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.530439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38f1bd-58fd-45df-9bd0-7b7986e2d7f2 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.610161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:32.631709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681575a8-34cb-42a9-b861-e39a427534f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.632402 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681575a8-34cb-42a9-b861-e39a427534f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.632774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681575a8-34cb-42a9-b861-e39a427534f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.633058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681575a8-34cb-42a9-b861-e39a427534f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.633754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681575a8-34cb-42a9-b861-e39a427534f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.635980 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681575a8-34cb-42a9-b861-e39a427534f0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.711091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1196/2411] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:40:32 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:32.712390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d252c7-5b91-403f-bc16-8d6cd5197f80 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.729899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d252c7-5b91-403f-bc16-8d6cd5197f80 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.735762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1288d2-1e1b-4818-a0bd-7bf6a16764e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.740579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1288d2-1e1b-4818-a0bd-7bf6a16764e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.740579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1288d2-1e1b-4818-a0bd-7bf6a16764e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.740579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1288d2-1e1b-4818-a0bd-7bf6a16764e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.740579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1288d2-1e1b-4818-a0bd-7bf6a16764e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.740579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1288d2-1e1b-4818-a0bd-7bf6a16764e1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.760159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1216/2412] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:32.762017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681575a8-34cb-42a9-b861-e39a427534f0 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.773403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681575a8-34cb-42a9-b861-e39a427534f0 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.778888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0a6f-0bdd-4569-8175-b81f64fd6193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.779851 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0a6f-0bdd-4569-8175-b81f64fd6193 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.780205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0a6f-0bdd-4569-8175-b81f64fd6193 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.780609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0a6f-0bdd-4569-8175-b81f64fd6193 admin admin] PATH_INFO: `/v3/projects/c75af4100981446e85cf7c3fb2900dcd/users/29d861f4214a43e3982009d1ff9f62e4/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.798016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e0a6f-0bdd-4569-8175-b81f64fd6193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75af4100981446e85cf7c3fb2900dcd, user_id=29d861f4214a43e3982009d1ff9f62e4, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.800778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e0a6f-0bdd-4569-8175-b81f64fd6193 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.859417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1197/2413] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:32 2022] PUT /identity/v3/projects/c75af4100981446e85cf7c3fb2900dcd/users/29d861f4214a43e3982009d1ff9f62e4/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:32.863326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1288d2-1e1b-4818-a0bd-7bf6a16764e1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.875003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1288d2-1e1b-4818-a0bd-7bf6a16764e1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.939658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babea9dc-4cbf-49eb-9f3b-54f0bc93ebed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.940722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babea9dc-4cbf-49eb-9f3b-54f0bc93ebed admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.941304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babea9dc-4cbf-49eb-9f3b-54f0bc93ebed admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.941728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babea9dc-4cbf-49eb-9f3b-54f0bc93ebed admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.942774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babea9dc-4cbf-49eb-9f3b-54f0bc93ebed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.944846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babea9dc-4cbf-49eb-9f3b-54f0bc93ebed admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.960634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1217/2414] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:32.960962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0a6f-0bdd-4569-8175-b81f64fd6193 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.971683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0a6f-0bdd-4569-8175-b81f64fd6193 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.978060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.978963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.979249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.979535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] PATH_INFO: `/v3/projects/c75af4100981446e85cf7c3fb2900dcd/users/29d861f4214a43e3982009d1ff9f62e4/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.981747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75af4100981446e85cf7c3fb2900dcd, user_id=29d861f4214a43e3982009d1ff9f62e4, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.984971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:33.000160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1198/2415] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:32 2022] PUT /identity/v3/projects/c75af4100981446e85cf7c3fb2900dcd/users/29d861f4214a43e3982009d1ff9f62e4/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:33.011548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0f7cd-2116-4cf5-88f0-14403a4dadb8 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:33.012070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0f7cd-2116-4cf5-88f0-14403a4dadb8 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:33.012432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0f7cd-2116-4cf5-88f0-14403a4dadb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:33.019719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:33.042855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4fe0f7cd-2116-4cf5-88f0-14403a4dadb8 None None] MFA Rules not processed for user `29d861f4214a43e3982009d1ff9f62e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:33.044180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4fe0f7cd-2116-4cf5-88f0-14403a4dadb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:33.137746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1218/2416] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:33.155178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:33.158207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:33.236025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f02f95-ed86-4e5b-ba4a-9e38259ef5ae admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:33.240160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488a6d5-32a7-4cd6-8b15-53c6a7f5144a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:33.240984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488a6d5-32a7-4cd6-8b15-53c6a7f5144a service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:33.242593 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488a6d5-32a7-4cd6-8b15-53c6a7f5144a service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:33.245995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488a6d5-32a7-4cd6-8b15-53c6a7f5144a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:33.254179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488a6d5-32a7-4cd6-8b15-53c6a7f5144a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:33.255723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488a6d5-32a7-4cd6-8b15-53c6a7f5144a service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:33.266315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1199/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:33.460952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0f7cd-2116-4cf5-88f0-14403a4dadb8 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:33.471722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0f7cd-2116-4cf5-88f0-14403a4dadb8 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:33.475557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:33.476200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:33.476490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:33.476798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:33.550388 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:33.551866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:33.563006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1219/2418] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:36.115320 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e2e66-065a-41df-a29c-7331efeec0ac None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:36.115790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e2e66-065a-41df-a29c-7331efeec0ac None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:36.116026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e2e66-065a-41df-a29c-7331efeec0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:36.134078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a72e2e66-065a-41df-a29c-7331efeec0ac None None] MFA Rules not processed for user `f2aa1b8204b1494589e096ad775ce4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:36.135284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a72e2e66-065a-41df-a29c-7331efeec0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:36.190490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1200/2419] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:36.435627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:36.437239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:36.494727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737893be-971b-4721-a38d-da314d67e6f3 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:36.498350 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65094a2-d92f-41ae-8e64-34db8e471a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:36.498932 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65094a2-d92f-41ae-8e64-34db8e471a28 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:36.499214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65094a2-d92f-41ae-8e64-34db8e471a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:36.499491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65094a2-d92f-41ae-8e64-34db8e471a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:36.501265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a65094a2-d92f-41ae-8e64-34db8e471a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:36.510808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65094a2-d92f-41ae-8e64-34db8e471a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:36.512225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65094a2-d92f-41ae-8e64-34db8e471a28 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:36.520751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1220/2420] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:36.917014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e2e66-065a-41df-a29c-7331efeec0ac None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:36.921744 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a72e2e66-065a-41df-a29c-7331efeec0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:36.980905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:37.006323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e2e66-065a-41df-a29c-7331efeec0ac None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:37.011896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:37.012667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:37.013020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:37.013433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:37.016542 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:37.028909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:37.030507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:37.041321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1201/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:41.980211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b026430-b212-488f-9294-ccd55842ab6d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:41.980694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b026430-b212-488f-9294-ccd55842ab6d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:41.980694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b026430-b212-488f-9294-ccd55842ab6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:42.005331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-3b026430-b212-488f-9294-ccd55842ab6d None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:42.007217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3b026430-b212-488f-9294-ccd55842ab6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:42.040913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:42.075088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1221/2422] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:40:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:42.087245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:42.088902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:42.159966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c9bc5-b480-4afa-8bc5-e7ee755adbbd service placement] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:42.166168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec4f4-e192-4079-8fc8-332446229cd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:42.167090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec4f4-e192-4079-8fc8-332446229cd7 service swift] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:42.167577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec4f4-e192-4079-8fc8-332446229cd7 service swift] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:42.168038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec4f4-e192-4079-8fc8-332446229cd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:42.180339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ec4f4-e192-4079-8fc8-332446229cd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:42.182319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ec4f4-e192-4079-8fc8-332446229cd7 service swift] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:42.193168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1202/2423] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:40:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:40:45.260429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b026430-b212-488f-9294-ccd55842ab6d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:45.263109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3b026430-b212-488f-9294-ccd55842ab6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:45.414563 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b026430-b212-488f-9294-ccd55842ab6d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:45.427227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd16c2f-7c1d-40c7-8415-993a837fe9c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:45.428053 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd16c2f-7c1d-40c7-8415-993a837fe9c1 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:45.428443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd16c2f-7c1d-40c7-8415-993a837fe9c1 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:45.428901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd16c2f-7c1d-40c7-8415-993a837fe9c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:45.431287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1fd16c2f-7c1d-40c7-8415-993a837fe9c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:45.461843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd16c2f-7c1d-40c7-8415-993a837fe9c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:45.463542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd16c2f-7c1d-40c7-8415-993a837fe9c1 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:45.476339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1222/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:45.531645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec4f4-e192-4079-8fc8-332446229cd7 service swift] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:45.633431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec4f4-e192-4079-8fc8-332446229cd7 service swift] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:45.645914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:45.647022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:45.647315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:45.647772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] PATH_INFO: `/v3/users/c9487779b7c24cf4a90ee61ae9a6a0c7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:45.649225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9487779b7c24cf4a90ee61ae9a6a0c7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:45.651261 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:45.869941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9487779b7c24cf4a90ee61ae9a6a0c7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:45.878268 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9487779b7c24cf4a90ee61ae9a6a0c7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:45.880588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9487779b7c24cf4a90ee61ae9a6a0c7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:45.880588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9487779b7c24cf4a90ee61ae9a6a0c7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:45.981042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9487779b7c24cf4a90ee61ae9a6a0c7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:45.982808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:46.001379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1203/2425] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:45 2022] DELETE /identity/v3/users/c9487779b7c24cf4a90ee61ae9a6a0c7 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:46.006344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1ed42-d280-4399-9ccc-3dea72440bf5 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:46.006718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1ed42-d280-4399-9ccc-3dea72440bf5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:46.007012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1ed42-d280-4399-9ccc-3dea72440bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:46.026722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f3e1ed42-d280-4399-9ccc-3dea72440bf5 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:46.030326 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f3e1ed42-d280-4399-9ccc-3dea72440bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:46.103794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1223/2426] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:46.116896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:46.119879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:46.222206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf24f-fb05-437c-8988-a9b94ee15744 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:46.226401 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169b561-92da-4194-ab80-19a677774c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:46.227436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169b561-92da-4194-ab80-19a677774c06 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:46.227672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169b561-92da-4194-ab80-19a677774c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:46.228002 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169b561-92da-4194-ab80-19a677774c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:46.238995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169b561-92da-4194-ab80-19a677774c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:46.242284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169b561-92da-4194-ab80-19a677774c06 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:46.270737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1204/2427] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:46.707973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1ed42-d280-4399-9ccc-3dea72440bf5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:46.709537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f3e1ed42-d280-4399-9ccc-3dea72440bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:46.733982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1ed42-d280-4399-9ccc-3dea72440bf5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:46.748927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:46.748927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:46.748927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:46.748927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] PATH_INFO: `/v3/projects/a9cf1ac6d8114d37b7658673777008ce` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:46.749341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9cf1ac6d8114d37b7658673777008ce)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:46.749341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:46.886409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9cf1ac6d8114d37b7658673777008ce', 'request_id': 'req-003253d3-ddfe-43d9-b051-7cae001754b9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:46.958783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9cf1ac6d8114d37b7658673777008ce', 'request_id': 'req-003253d3-ddfe-43d9-b051-7cae001754b9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:46.960503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:46.967428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9cf1ac6d8114d37b7658673777008ce', 'request_id': 'req-003253d3-ddfe-43d9-b051-7cae001754b9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:46.980386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] The token cache is being invalidate because project a9cf1ac6d8114d37b7658673777008ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:46.980799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:46.981651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:46.986192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1224/2428] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:46 2022] DELETE /identity/v3/projects/a9cf1ac6d8114d37b7658673777008ce => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:46.987561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169b561-92da-4194-ab80-19a677774c06 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:46.999387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169b561-92da-4194-ab80-19a677774c06 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:47.081236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:47.082268 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:47.082555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:47.082920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] PATH_INFO: `/v3/users/29d861f4214a43e3982009d1ff9f62e4` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:47.084149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d861f4214a43e3982009d1ff9f62e4)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:47.086276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:47.264656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d861f4214a43e3982009d1ff9f62e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:47.283709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d861f4214a43e3982009d1ff9f62e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:47.290830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d861f4214a43e3982009d1ff9f62e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:47.300978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d861f4214a43e3982009d1ff9f62e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:47.408290 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-135aee40-5d38-4301-ba4e-9781ee678056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d861f4214a43e3982009d1ff9f62e4'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:47.434613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1205/2429] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:46 2022] DELETE /identity/v3/users/29d861f4214a43e3982009d1ff9f62e4 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:47.959842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:47.962049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:47.980366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003253d3-ddfe-43d9-b051-7cae001754b9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:48.056953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:48.057870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.058368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.058652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] PATH_INFO: `/v3/projects/c75af4100981446e85cf7c3fb2900dcd` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.059868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75af4100981446e85cf7c3fb2900dcd)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:48.061829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:48.173729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75af4100981446e85cf7c3fb2900dcd', 'request_id': 'req-864ab1c8-3f5e-4370-8544-ca99d107c138'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:48.266978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75af4100981446e85cf7c3fb2900dcd', 'request_id': 'req-864ab1c8-3f5e-4370-8544-ca99d107c138'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:48.274781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75af4100981446e85cf7c3fb2900dcd', 'request_id': 'req-864ab1c8-3f5e-4370-8544-ca99d107c138'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:48.300560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] The token cache is being invalidate because project c75af4100981446e85cf7c3fb2900dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:48.301318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:48.301997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-864ab1c8-3f5e-4370-8544-ca99d107c138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:48.304142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1225/2430] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:47 2022] DELETE /identity/v3/projects/c75af4100981446e85cf7c3fb2900dcd => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:48.309280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae13d2-e8d3-4b55-a49c-0fddc2a0ff16 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.309843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae13d2-e8d3-4b55-a49c-0fddc2a0ff16 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.309843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae13d2-e8d3-4b55-a49c-0fddc2a0ff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.316814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:48.330196 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-adae13d2-e8d3-4b55-a49c-0fddc2a0ff16 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:48.331442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-adae13d2-e8d3-4b55-a49c-0fddc2a0ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:48.406052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1206/2431] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:48.414295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089a062-1854-4dcf-aa51-f5048d6ae0e6 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.414551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089a062-1854-4dcf-aa51-f5048d6ae0e6 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.414836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089a062-1854-4dcf-aa51-f5048d6ae0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.438457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2089a062-1854-4dcf-aa51-f5048d6ae0e6 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:48.439694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2089a062-1854-4dcf-aa51-f5048d6ae0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:48.448798 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1226/2432] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:48.454372 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae13d2-e8d3-4b55-a49c-0fddc2a0ff16 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:48.468503 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae13d2-e8d3-4b55-a49c-0fddc2a0ff16 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:48.475371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:48.476332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.476723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.477088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.477912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:40:48.478302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:48.480049 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:48.495841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1207/2433] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:40:48.498780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089a062-1854-4dcf-aa51-f5048d6ae0e6 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:48.510071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089a062-1854-4dcf-aa51-f5048d6ae0e6 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:48.515567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32e967-f9d5-41b6-9cde-5b1159f66760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:48.516241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32e967-f9d5-41b6-9cde-5b1159f66760 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.516551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32e967-f9d5-41b6-9cde-5b1159f66760 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.516844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32e967-f9d5-41b6-9cde-5b1159f66760 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.517776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32e967-f9d5-41b6-9cde-5b1159f66760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:48.519682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32e967-f9d5-41b6-9cde-5b1159f66760 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:48.606644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1227/2434] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:48 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:48.611349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:48.627721 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54a3f1-9d88-4f25-a3dd-dc3defbfdb16 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:48.701723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f41d2-b8b3-4c19-a577-0b4fe4a5e27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:48.702528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f41d2-b8b3-4c19-a577-0b4fe4a5e27c admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.702888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f41d2-b8b3-4c19-a577-0b4fe4a5e27c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.703271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f41d2-b8b3-4c19-a577-0b4fe4a5e27c admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.704042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f41d2-b8b3-4c19-a577-0b4fe4a5e27c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:48.706151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f41d2-b8b3-4c19-a577-0b4fe4a5e27c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:48.810769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1208/2435] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:40:48 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:48.815037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32e967-f9d5-41b6-9cde-5b1159f66760 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:48.826634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32e967-f9d5-41b6-9cde-5b1159f66760 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:48.848289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379be18-5014-4d7e-b625-66da9aabcdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:48.852612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379be18-5014-4d7e-b625-66da9aabcdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.852928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379be18-5014-4d7e-b625-66da9aabcdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.853357 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379be18-5014-4d7e-b625-66da9aabcdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.854244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379be18-5014-4d7e-b625-66da9aabcdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:48.855913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379be18-5014-4d7e-b625-66da9aabcdb5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:48.877801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1228/2436] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:48.881736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f41d2-b8b3-4c19-a577-0b4fe4a5e27c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:48.893775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f41d2-b8b3-4c19-a577-0b4fe4a5e27c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:48.900109 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0059bfd-f1cb-44e1-9582-98f57ee2f06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:48.900916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059bfd-f1cb-44e1-9582-98f57ee2f06e admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.901265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059bfd-f1cb-44e1-9582-98f57ee2f06e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.901609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059bfd-f1cb-44e1-9582-98f57ee2f06e admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.902418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0059bfd-f1cb-44e1-9582-98f57ee2f06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:48.904079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0059bfd-f1cb-44e1-9582-98f57ee2f06e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:48.918879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1209/2437] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:40:48.920139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379be18-5014-4d7e-b625-66da9aabcdb5 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:48.932002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379be18-5014-4d7e-b625-66da9aabcdb5 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:48.937406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc442d-44d6-48c4-84ce-0841f76bc723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:48.938651 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc442d-44d6-48c4-84ce-0841f76bc723 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:48.939037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc442d-44d6-48c4-84ce-0841f76bc723 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:48.939521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc442d-44d6-48c4-84ce-0841f76bc723 admin admin] PATH_INFO: `/v3/projects/100ed615e745438fa184c9c822a4fe52/users/bf90226ea0644852a6c2ec103d8fcbd1/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:48.958492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc442d-44d6-48c4-84ce-0841f76bc723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100ed615e745438fa184c9c822a4fe52, user_id=bf90226ea0644852a6c2ec103d8fcbd1, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:48.961538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc442d-44d6-48c4-84ce-0841f76bc723 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:49.049755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1229/2438] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:48 2022] PUT /identity/v3/projects/100ed615e745438fa184c9c822a4fe52/users/bf90226ea0644852a6c2ec103d8fcbd1/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:49.051605 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790f86-034d-4060-8c83-d6e0923b42bd None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:49.051894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790f86-034d-4060-8c83-d6e0923b42bd None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:49.052176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790f86-034d-4060-8c83-d6e0923b42bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:49.080319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f3790f86-034d-4060-8c83-d6e0923b42bd None None] MFA Rules not processed for user `bf90226ea0644852a6c2ec103d8fcbd1`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:49.081925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f3790f86-034d-4060-8c83-d6e0923b42bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:49.171707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:49.177582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1210/2439] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:49.183692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc442d-44d6-48c4-84ce-0841f76bc723 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:49.197079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:49.274786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc442d-44d6-48c4-84ce-0841f76bc723 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:49.281167 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6b6e8-9c6c-43ef-9111-a6e63369fd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:49.285813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6b6e8-9c6c-43ef-9111-a6e63369fd53 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:49.285813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6b6e8-9c6c-43ef-9111-a6e63369fd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:49.285813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6b6e8-9c6c-43ef-9111-a6e63369fd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:49.285813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6b6e8-9c6c-43ef-9111-a6e63369fd53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:49.285813 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6b6e8-9c6c-43ef-9111-a6e63369fd53 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:49.503280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1230/2440] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:49 2022] POST /identity/v3/projects => generated 379 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:40:49.508294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3790f86-034d-4060-8c83-d6e0923b42bd None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:49.531623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3790f86-034d-4060-8c83-d6e0923b42bd None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:49.608578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee1fca-e41c-4311-8852-aa27256a826c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:49.609261 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee1fca-e41c-4311-8852-aa27256a826c admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:49.609589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee1fca-e41c-4311-8852-aa27256a826c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:49.609869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee1fca-e41c-4311-8852-aa27256a826c admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:49.610576 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee1fca-e41c-4311-8852-aa27256a826c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:49.612498 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee1fca-e41c-4311-8852-aa27256a826c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:49.717665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1211/2441] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:40:49 2022] POST /identity/v3/users => generated 296 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:49.722115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6b6e8-9c6c-43ef-9111-a6e63369fd53 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:49.735455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6b6e8-9c6c-43ef-9111-a6e63369fd53 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:49.750920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd488b39-5cb0-4f61-abf9-f6e94536ac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:49.752668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd488b39-5cb0-4f61-abf9-f6e94536ac62 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:49.754640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd488b39-5cb0-4f61-abf9-f6e94536ac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:49.755491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd488b39-5cb0-4f61-abf9-f6e94536ac62 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:49.758970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd488b39-5cb0-4f61-abf9-f6e94536ac62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:49.760717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd488b39-5cb0-4f61-abf9-f6e94536ac62 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:49.785452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1231/2442] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:49.789322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee1fca-e41c-4311-8852-aa27256a826c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:49.800983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee1fca-e41c-4311-8852-aa27256a826c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:49.805651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65908db4-f0e3-43f3-9214-1cf11f963671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:49.806546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65908db4-f0e3-43f3-9214-1cf11f963671 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:49.806830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65908db4-f0e3-43f3-9214-1cf11f963671 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:49.807114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65908db4-f0e3-43f3-9214-1cf11f963671 admin admin] PATH_INFO: `/v3/projects/11a2c65a94ff4c739e24eaf9b0d2be54/users/7b5bf4b223294b328f1db51eea3d240e/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:49.822954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65908db4-f0e3-43f3-9214-1cf11f963671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a2c65a94ff4c739e24eaf9b0d2be54, user_id=7b5bf4b223294b328f1db51eea3d240e, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:49.825820 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65908db4-f0e3-43f3-9214-1cf11f963671 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:49.922436 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1212/2443] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:49 2022] PUT /identity/v3/projects/11a2c65a94ff4c739e24eaf9b0d2be54/users/7b5bf4b223294b328f1db51eea3d240e/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:49.923525 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd488b39-5cb0-4f61-abf9-f6e94536ac62 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:49.935203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd488b39-5cb0-4f61-abf9-f6e94536ac62 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:49.996914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8e0c-12e3-4b9a-801b-04293cd15f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:49.997693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8e0c-12e3-4b9a-801b-04293cd15f36 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:49.998057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8e0c-12e3-4b9a-801b-04293cd15f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:49.998341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8e0c-12e3-4b9a-801b-04293cd15f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:49.999085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8e0c-12e3-4b9a-801b-04293cd15f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:50.001345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8e0c-12e3-4b9a-801b-04293cd15f36 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:50.023908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1232/2444] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:50.025840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65908db4-f0e3-43f3-9214-1cf11f963671 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:50.037017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65908db4-f0e3-43f3-9214-1cf11f963671 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:50.041680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:50.042688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:50.042974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:50.043257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] PATH_INFO: `/v3/projects/11a2c65a94ff4c739e24eaf9b0d2be54/users/7b5bf4b223294b328f1db51eea3d240e/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:50.045292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a2c65a94ff4c739e24eaf9b0d2be54, user_id=7b5bf4b223294b328f1db51eea3d240e, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:50.048482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:50.059476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1213/2445] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:50 2022] PUT /identity/v3/projects/11a2c65a94ff4c739e24eaf9b0d2be54/users/7b5bf4b223294b328f1db51eea3d240e/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:50.071803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52fd6f-de5d-40c0-9589-ec99d35d5441 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:50.072103 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52fd6f-de5d-40c0-9589-ec99d35d5441 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:50.072510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52fd6f-de5d-40c0-9589-ec99d35d5441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:50.101172 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6b52fd6f-de5d-40c0-9589-ec99d35d5441 None None] MFA Rules not processed for user `7b5bf4b223294b328f1db51eea3d240e`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:50.102402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6b52fd6f-de5d-40c0-9589-ec99d35d5441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:50.171427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1233/2446] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:50.183962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:50.186629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:50.244881 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6aea4c-3bbf-414b-891f-5656119c8b72 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:50.248551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9ed8-c5f1-4208-a39f-3a48c67c1816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:50.249209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9ed8-c5f1-4208-a39f-3a48c67c1816 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:50.249496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9ed8-c5f1-4208-a39f-3a48c67c1816 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:50.249779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9ed8-c5f1-4208-a39f-3a48c67c1816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:50.259326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f9ed8-c5f1-4208-a39f-3a48c67c1816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:50.260777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f9ed8-c5f1-4208-a39f-3a48c67c1816 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:50.270142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1214/2447] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:51.315070 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52fd6f-de5d-40c0-9589-ec99d35d5441 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:51.325727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52fd6f-de5d-40c0-9589-ec99d35d5441 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:51.342223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b5bf4b223294b328f1db51eea3d240e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11a2c65a94ff4c739e24eaf9b0d2be54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:51.342223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:51.342223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:51.342541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:51.342541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:40:51.342541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:51.342541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:51.342541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:51.352794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1234/2448] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:40:51.365339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9ed8-c5f1-4208-a39f-3a48c67c1816 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:51.374820 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9ed8-c5f1-4208-a39f-3a48c67c1816 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:51.380274 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c707b-22d1-46c0-9967-1752a473e734 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b5bf4b223294b328f1db51eea3d240e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11a2c65a94ff4c739e24eaf9b0d2be54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:51.381045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c707b-22d1-46c0-9967-1752a473e734 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:51.381412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c707b-22d1-46c0-9967-1752a473e734 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:51.381769 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c707b-22d1-46c0-9967-1752a473e734 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:51.383093 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c707b-22d1-46c0-9967-1752a473e734 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:51.384454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c707b-22d1-46c0-9967-1752a473e734 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:51.476657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1215/2449] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:51 2022] POST /identity/v3/projects => generated 354 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:51.480308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:51.493203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbc7dc-c7be-4fc4-a8af-2c2f7b050a46 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:51.565981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b5bf4b223294b328f1db51eea3d240e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11a2c65a94ff4c739e24eaf9b0d2be54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:51.566738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:51.566961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:51.567252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:51.568000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:40:51.568328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:51.570302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:51.590249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1235/2450] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:40:51.620775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc64cb-e2c1-43fc-bbb2-b20bf39e7788 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:51.621067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc64cb-e2c1-43fc-bbb2-b20bf39e7788 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:51.621343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc64cb-e2c1-43fc-bbb2-b20bf39e7788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:51.631753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:51.644678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be952d7f-f164-40a7-9a97-66e8d29a617d tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:51.645115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-3edc64cb-e2c1-43fc-bbb2-b20bf39e7788 None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:51.646594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3edc64cb-e2c1-43fc-bbb2-b20bf39e7788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:51.649836 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b5bf4b223294b328f1db51eea3d240e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11a2c65a94ff4c739e24eaf9b0d2be54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:51.652811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:51.652811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:51.652811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] PATH_INFO: `/v3/projects/1436dc25a999459bb9c23152d38153c4` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:51.655028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1436dc25a999459bb9c23152d38153c4)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:51.657275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:51.687239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:51.724795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1216/2451] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:40:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:51.743031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc64cb-e2c1-43fc-bbb2-b20bf39e7788 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:51.743031 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3edc64cb-e2c1-43fc-bbb2-b20bf39e7788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:51.808361 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1436dc25a999459bb9c23152d38153c4', 'request_id': 'req-13836613-7567-401c-8f9d-847eb6bf5258'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:51.825521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc64cb-e2c1-43fc-bbb2-b20bf39e7788 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:51.896184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad7f45-cdd5-4c3c-9957-da21ded9f82e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:51.896882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad7f45-cdd5-4c3c-9957-da21ded9f82e service swift] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:51.897198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad7f45-cdd5-4c3c-9957-da21ded9f82e service swift] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:51.897477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad7f45-cdd5-4c3c-9957-da21ded9f82e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:51.906871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1436dc25a999459bb9c23152d38153c4', 'request_id': 'req-13836613-7567-401c-8f9d-847eb6bf5258'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:51.917503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1436dc25a999459bb9c23152d38153c4', 'request_id': 'req-13836613-7567-401c-8f9d-847eb6bf5258'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:51.932975 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] The token cache is being invalidate because project 1436dc25a999459bb9c23152d38153c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:51.933346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:51.934519 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:51.935838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1236/2452] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:51 2022] DELETE /identity/v3/projects/1436dc25a999459bb9c23152d38153c4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:51.963409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad7f45-cdd5-4c3c-9957-da21ded9f82e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:51.965069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad7f45-cdd5-4c3c-9957-da21ded9f82e service swift] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:51.975202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1217/2453] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:40:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:40:53.672890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:53.674414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:53.774417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13836613-7567-401c-8f9d-847eb6bf5258 tempest-VolumeQuotaClassesTest-344912346 tempest-VolumeQuotaClassesTest-344912346-project-admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:53.776814 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:53.777701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:53.777989 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:53.778271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] PATH_INFO: `/v3/users/bf90226ea0644852a6c2ec103d8fcbd1` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:53.779474 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf90226ea0644852a6c2ec103d8fcbd1)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:53.781615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:53.799720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:54.038847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf90226ea0644852a6c2ec103d8fcbd1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:54.054811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf90226ea0644852a6c2ec103d8fcbd1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:54.065681 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf90226ea0644852a6c2ec103d8fcbd1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:54.066693 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf90226ea0644852a6c2ec103d8fcbd1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:54.067546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf90226ea0644852a6c2ec103d8fcbd1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:54.135628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1237/2454] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:53 2022] DELETE /identity/v3/users/bf90226ea0644852a6c2ec103d8fcbd1 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:40:54.140662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c9f13-0d5a-46f4-8385-de3c8eea0b5e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:54.142205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c9f13-0d5a-46f4-8385-de3c8eea0b5e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:54.142811 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c9f13-0d5a-46f4-8385-de3c8eea0b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:54.161864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-b97c9f13-0d5a-46f4-8385-de3c8eea0b5e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:54.163407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b97c9f13-0d5a-46f4-8385-de3c8eea0b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:54.178095 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:54.239776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1218/2455] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:40:54.249614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:54.251549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:54.314675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f1698-e4ac-4b76-889f-1f9a2fe6c999 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:54.318315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd03112-a7ba-496f-a9d4-32489d936b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:54.319227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd03112-a7ba-496f-a9d4-32489d936b72 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:54.319558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd03112-a7ba-496f-a9d4-32489d936b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:54.320028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd03112-a7ba-496f-a9d4-32489d936b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:54.331198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd03112-a7ba-496f-a9d4-32489d936b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:54.332663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd03112-a7ba-496f-a9d4-32489d936b72 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:54.350356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1238/2456] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:55.238839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c9f13-0d5a-46f4-8385-de3c8eea0b5e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:55.240370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b97c9f13-0d5a-46f4-8385-de3c8eea0b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:55.255748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c9f13-0d5a-46f4-8385-de3c8eea0b5e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:55.260825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:55.261810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:55.262143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:55.262425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] PATH_INFO: `/v3/projects/100ed615e745438fa184c9c822a4fe52` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:55.263627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100ed615e745438fa184c9c822a4fe52)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:55.266080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:55.366230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100ed615e745438fa184c9c822a4fe52', 'request_id': 'req-19799e73-a8a7-4e44-ae36-226fc871b017'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.430842 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100ed615e745438fa184c9c822a4fe52', 'request_id': 'req-19799e73-a8a7-4e44-ae36-226fc871b017'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.443723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100ed615e745438fa184c9c822a4fe52', 'request_id': 'req-19799e73-a8a7-4e44-ae36-226fc871b017'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.460806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] The token cache is being invalidate because project 100ed615e745438fa184c9c822a4fe52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:55.461094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.461760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.463643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1219/2457] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:55 2022] DELETE /identity/v3/projects/100ed615e745438fa184c9c822a4fe52 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:55.469709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd03112-a7ba-496f-a9d4-32489d936b72 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:55.480493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd03112-a7ba-496f-a9d4-32489d936b72 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:55.552051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:55.552974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:55.553261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:55.559050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] PATH_INFO: `/v3/users/7b5bf4b223294b328f1db51eea3d240e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:55.559961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5bf4b223294b328f1db51eea3d240e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:55.563029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:55.752166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:55.768126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5bf4b223294b328f1db51eea3d240e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.789520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5bf4b223294b328f1db51eea3d240e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.799925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5bf4b223294b328f1db51eea3d240e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.801110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5bf4b223294b328f1db51eea3d240e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.802321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1dcd134c-7cc1-40c4-8481-0546a64a076d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5bf4b223294b328f1db51eea3d240e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:55.896351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1239/2458] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:55 2022] DELETE /identity/v3/users/7b5bf4b223294b328f1db51eea3d240e => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:56.433411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:56.435396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:56.465920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19799e73-a8a7-4e44-ae36-226fc871b017 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:56.550377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:56.551381 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:56.551733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:56.552089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] PATH_INFO: `/v3/projects/11a2c65a94ff4c739e24eaf9b0d2be54` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:56.553417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11a2c65a94ff4c739e24eaf9b0d2be54)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:56.555531 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:56.681445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:56.683506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11a2c65a94ff4c739e24eaf9b0d2be54', 'request_id': 'req-344c694a-6e8a-41c0-a564-4b77e7037251'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:56.757527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11a2c65a94ff4c739e24eaf9b0d2be54', 'request_id': 'req-344c694a-6e8a-41c0-a564-4b77e7037251'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:56.768072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11a2c65a94ff4c739e24eaf9b0d2be54', 'request_id': 'req-344c694a-6e8a-41c0-a564-4b77e7037251'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:56.780615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] The token cache is being invalidate because project 11a2c65a94ff4c739e24eaf9b0d2be54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:56.780615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:56.780615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-344c694a-6e8a-41c0-a564-4b77e7037251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:56.788780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1220/2459] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:56 2022] DELETE /identity/v3/projects/11a2c65a94ff4c739e24eaf9b0d2be54 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:56.789149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a151b-af8b-4f81-b68b-a78e211250ff None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:56.789149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a151b-af8b-4f81-b68b-a78e211250ff None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:56.789149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a151b-af8b-4f81-b68b-a78e211250ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:56.808760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-cb9a151b-af8b-4f81-b68b-a78e211250ff None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:56.810006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-cb9a151b-af8b-4f81-b68b-a78e211250ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:56.899270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1240/2460] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:56.909219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb586-09a1-4ca5-a578-55826071297a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:56.909792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb586-09a1-4ca5-a578-55826071297a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:56.910899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb586-09a1-4ca5-a578-55826071297a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:56.941411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e41bb586-09a1-4ca5-a578-55826071297a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:56.942656 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e41bb586-09a1-4ca5-a578-55826071297a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:56.956015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1221/2461] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:56.963066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a151b-af8b-4f81-b68b-a78e211250ff None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:56.975280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a151b-af8b-4f81-b68b-a78e211250ff None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:56.985723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:56.986509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:56.986860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:56.987202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:56.988023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:40:56.988417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:56.990138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:57.009940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1241/2462] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:40:57.010809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb586-09a1-4ca5-a578-55826071297a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:57.026424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb586-09a1-4ca5-a578-55826071297a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:57.037967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22dcaf-ddf8-424a-aa3a-6b419a6dc1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:57.039556 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22dcaf-ddf8-424a-aa3a-6b419a6dc1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:57.040071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22dcaf-ddf8-424a-aa3a-6b419a6dc1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:57.040467 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22dcaf-ddf8-424a-aa3a-6b419a6dc1c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:57.041292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22dcaf-ddf8-424a-aa3a-6b419a6dc1c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:57.043651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22dcaf-ddf8-424a-aa3a-6b419a6dc1c7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:57.196697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1222/2463] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:57 2022] POST /identity/v3/projects => generated 389 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:57.200970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:57.217699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b3cdf-5b88-467b-a005-1c67c8945ff6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:57.299564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92436e-3daa-4637-95a8-03b8b5465080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:57.300392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92436e-3daa-4637-95a8-03b8b5465080 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:57.300811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92436e-3daa-4637-95a8-03b8b5465080 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:57.301193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92436e-3daa-4637-95a8-03b8b5465080 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:57.302030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92436e-3daa-4637-95a8-03b8b5465080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:57.304126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92436e-3daa-4637-95a8-03b8b5465080 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:57.394812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1242/2464] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:40:57 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:57.399561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22dcaf-ddf8-424a-aa3a-6b419a6dc1c7 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:57.412067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22dcaf-ddf8-424a-aa3a-6b419a6dc1c7 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:57.426087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac3ce5-e99e-464c-900a-f1a450cf6ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:57.429126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac3ce5-e99e-464c-900a-f1a450cf6ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:57.433294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac3ce5-e99e-464c-900a-f1a450cf6ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:57.433294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac3ce5-e99e-464c-900a-f1a450cf6ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:57.433294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac3ce5-e99e-464c-900a-f1a450cf6ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:57.435915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac3ce5-e99e-464c-900a-f1a450cf6ba6 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:57.451446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1223/2465] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:57.455275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92436e-3daa-4637-95a8-03b8b5465080 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:57.502188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92436e-3daa-4637-95a8-03b8b5465080 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:57.502188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feb002-31ca-4eb3-bf64-12ef3679ac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:57.502188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feb002-31ca-4eb3-bf64-12ef3679ac7a admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:57.502188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feb002-31ca-4eb3-bf64-12ef3679ac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:57.520423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feb002-31ca-4eb3-bf64-12ef3679ac7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:57.520423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feb002-31ca-4eb3-bf64-12ef3679ac7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:57.520423 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feb002-31ca-4eb3-bf64-12ef3679ac7a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:57.546786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1243/2466] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:57.549538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac3ce5-e99e-464c-900a-f1a450cf6ba6 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:57.569555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac3ce5-e99e-464c-900a-f1a450cf6ba6 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:57.595098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4bae-b3cc-49c9-b1ed-5df9a2e42008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:57.595098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4bae-b3cc-49c9-b1ed-5df9a2e42008 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:57.595098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4bae-b3cc-49c9-b1ed-5df9a2e42008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:57.595098 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4bae-b3cc-49c9-b1ed-5df9a2e42008 admin admin] PATH_INFO: `/v3/projects/9d4515e63c6f4a339b9ef3d53a5f245b/users/78a287e1b1dd4723965447788bea42f7/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:57.627646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d4bae-b3cc-49c9-b1ed-5df9a2e42008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d4515e63c6f4a339b9ef3d53a5f245b, user_id=78a287e1b1dd4723965447788bea42f7, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:57.630495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d4bae-b3cc-49c9-b1ed-5df9a2e42008 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:57.707567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1224/2467] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:57 2022] PUT /identity/v3/projects/9d4515e63c6f4a339b9ef3d53a5f245b/users/78a287e1b1dd4723965447788bea42f7/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:57.718929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acb306-e152-43b1-887e-9f7bda65c482 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:57.719216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acb306-e152-43b1-887e-9f7bda65c482 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:57.719557 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acb306-e152-43b1-887e-9f7bda65c482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:57.788441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e0acb306-e152-43b1-887e-9f7bda65c482 None None] MFA Rules not processed for user `78a287e1b1dd4723965447788bea42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:57.789696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e0acb306-e152-43b1-887e-9f7bda65c482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:57.798608 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:57.897708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1244/2468] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:57.902307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4bae-b3cc-49c9-b1ed-5df9a2e42008 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:57.988348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4bae-b3cc-49c9-b1ed-5df9a2e42008 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:57.993036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545025-bba9-4c33-9f47-e629efe6533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:57.993714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545025-bba9-4c33-9f47-e629efe6533c admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:57.993996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545025-bba9-4c33-9f47-e629efe6533c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:57.994273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545025-bba9-4c33-9f47-e629efe6533c admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:57.994978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff545025-bba9-4c33-9f47-e629efe6533c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:57.996913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff545025-bba9-4c33-9f47-e629efe6533c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:58.456245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1225/2469] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:40:57 2022] POST /identity/v3/projects => generated 387 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:58.473402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acb306-e152-43b1-887e-9f7bda65c482 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:58.483835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acb306-e152-43b1-887e-9f7bda65c482 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:58.540266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5617-cde0-4596-9248-dcfe9c8f2122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:58.540963 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5617-cde0-4596-9248-dcfe9c8f2122 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:58.541243 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5617-cde0-4596-9248-dcfe9c8f2122 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:58.541520 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5617-cde0-4596-9248-dcfe9c8f2122 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:58.542207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a5617-cde0-4596-9248-dcfe9c8f2122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:58.544107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a5617-cde0-4596-9248-dcfe9c8f2122 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:58.627931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1245/2470] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:40:58 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:40:58.633981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545025-bba9-4c33-9f47-e629efe6533c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:58.646146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545025-bba9-4c33-9f47-e629efe6533c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:58.650877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367ac94-87e9-4e9f-bfc8-5f2765c4444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:58.651546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ac94-87e9-4e9f-bfc8-5f2765c4444b admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:58.651831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ac94-87e9-4e9f-bfc8-5f2765c4444b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:58.655618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ac94-87e9-4e9f-bfc8-5f2765c4444b admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:58.657813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367ac94-87e9-4e9f-bfc8-5f2765c4444b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:58.659458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367ac94-87e9-4e9f-bfc8-5f2765c4444b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:58.676704 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5617-cde0-4596-9248-dcfe9c8f2122 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:58.678546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1226/2471] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:58.690258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5617-cde0-4596-9248-dcfe9c8f2122 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:58.698315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f784a-40c8-4497-8f60-cd504f1cb3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:58.700395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f784a-40c8-4497-8f60-cd504f1cb3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:58.700395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f784a-40c8-4497-8f60-cd504f1cb3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:58.700395 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f784a-40c8-4497-8f60-cd504f1cb3d4 admin admin] PATH_INFO: `/v3/projects/a9a05980bcc34cf1a771863248627673/users/dda3f141c850493885c86dec06006c27/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:58.726370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f784a-40c8-4497-8f60-cd504f1cb3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a05980bcc34cf1a771863248627673, user_id=dda3f141c850493885c86dec06006c27, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:58.730476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f784a-40c8-4497-8f60-cd504f1cb3d4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:58.850229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1246/2472] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:58 2022] PUT /identity/v3/projects/a9a05980bcc34cf1a771863248627673/users/dda3f141c850493885c86dec06006c27/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:40:58.854732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367ac94-87e9-4e9f-bfc8-5f2765c4444b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:58.871432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367ac94-87e9-4e9f-bfc8-5f2765c4444b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:58.933839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:58.969514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d056dd5-de9e-483b-9c39-c8a001b82e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:58.971537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056dd5-de9e-483b-9c39-c8a001b82e0d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:58.971537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056dd5-de9e-483b-9c39-c8a001b82e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:58.971537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056dd5-de9e-483b-9c39-c8a001b82e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:58.971537 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d056dd5-de9e-483b-9c39-c8a001b82e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:58.973198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d056dd5-de9e-483b-9c39-c8a001b82e0d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:58.988233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1227/2473] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:40:58.989379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f784a-40c8-4497-8f60-cd504f1cb3d4 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:58.999870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f784a-40c8-4497-8f60-cd504f1cb3d4 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:59.008797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:59.010069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:59.010414 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:59.010768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] PATH_INFO: `/v3/projects/a9a05980bcc34cf1a771863248627673/users/dda3f141c850493885c86dec06006c27/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:59.012908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a05980bcc34cf1a771863248627673, user_id=dda3f141c850493885c86dec06006c27, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:59.019034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:59.027536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1247/2474] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:40:58 2022] PUT /identity/v3/projects/a9a05980bcc34cf1a771863248627673/users/dda3f141c850493885c86dec06006c27/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:59.045797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb415226-53c3-4add-86fb-7c5daa462e8f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:59.046256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb415226-53c3-4add-86fb-7c5daa462e8f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:59.046647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb415226-53c3-4add-86fb-7c5daa462e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:59.083232 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-cb415226-53c3-4add-86fb-7c5daa462e8f None None] MFA Rules not processed for user `dda3f141c850493885c86dec06006c27`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:59.084693 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cb415226-53c3-4add-86fb-7c5daa462e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:59.175129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1228/2475] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:40:59.182769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:59.184758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:59.264779 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037af7ac-3b41-4db0-9f20-32431f3e69df admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:59.270628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c25f3-b8bc-434d-9642-0b0311b088db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:59.271862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c25f3-b8bc-434d-9642-0b0311b088db service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:59.271862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c25f3-b8bc-434d-9642-0b0311b088db service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:59.271862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c25f3-b8bc-434d-9642-0b0311b088db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:59.284570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c25f3-b8bc-434d-9642-0b0311b088db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:59.286493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c25f3-b8bc-434d-9642-0b0311b088db service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:59.301938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1248/2476] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:59.510281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb415226-53c3-4add-86fb-7c5daa462e8f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:59.597118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb415226-53c3-4add-86fb-7c5daa462e8f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:59.606423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:59.607627 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:59.608080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:59.608654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] PATH_INFO: `/v3/users/78a287e1b1dd4723965447788bea42f7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:59.611050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a287e1b1dd4723965447788bea42f7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:59.613641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:59.849995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a287e1b1dd4723965447788bea42f7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:59.859269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a287e1b1dd4723965447788bea42f7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:59.861004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a287e1b1dd4723965447788bea42f7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:59.861816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a287e1b1dd4723965447788bea42f7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:59.864794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:59.924035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a287e1b1dd4723965447788bea42f7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:59.955460 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1229/2477] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:59 2022] DELETE /identity/v3/users/78a287e1b1dd4723965447788bea42f7 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:59.961066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5ea31-37ca-45ac-963c-05e6c0ca83e1 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:59.961580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5ea31-37ca-45ac-963c-05e6c0ca83e1 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:59.961833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5ea31-37ca-45ac-963c-05e6c0ca83e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:59.968340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:59.983311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-63e5ea31-37ca-45ac-963c-05e6c0ca83e1 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:40:59.986071 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-63e5ea31-37ca-45ac-963c-05e6c0ca83e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:00.070719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1249/2478] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:00.080636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:00.082089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:00.177322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99073842-5be4-485c-8861-597245a1fb63 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:00.181463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1149-66bb-474f-8d40-be2ea7bf4701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:00.182513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1149-66bb-474f-8d40-be2ea7bf4701 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:00.182787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1149-66bb-474f-8d40-be2ea7bf4701 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:00.183072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1149-66bb-474f-8d40-be2ea7bf4701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:00.193697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b1149-66bb-474f-8d40-be2ea7bf4701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:00.194877 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b1149-66bb-474f-8d40-be2ea7bf4701 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:00.226849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1230/2479] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:00.721297 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5ea31-37ca-45ac-963c-05e6c0ca83e1 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:00.723436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-63e5ea31-37ca-45ac-963c-05e6c0ca83e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:00.746158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5ea31-37ca-45ac-963c-05e6c0ca83e1 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:00.763964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:00.765198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:00.766196 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:00.766661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] PATH_INFO: `/v3/projects/9d4515e63c6f4a339b9ef3d53a5f245b` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:00.768975 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d4515e63c6f4a339b9ef3d53a5f245b)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:00.771304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:00.888607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d4515e63c6f4a339b9ef3d53a5f245b', 'request_id': 'req-ee2dd1c3-704e-4dda-ad9e-7f709e181541'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.003578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d4515e63c6f4a339b9ef3d53a5f245b', 'request_id': 'req-ee2dd1c3-704e-4dda-ad9e-7f709e181541'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.011332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d4515e63c6f4a339b9ef3d53a5f245b', 'request_id': 'req-ee2dd1c3-704e-4dda-ad9e-7f709e181541'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.027385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] The token cache is being invalidate because project 9d4515e63c6f4a339b9ef3d53a5f245b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:01.027776 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.028660 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.030805 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1250/2480] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:00 2022] DELETE /identity/v3/projects/9d4515e63c6f4a339b9ef3d53a5f245b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:01.034678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1149-66bb-474f-8d40-be2ea7bf4701 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:01.052694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1149-66bb-474f-8d40-be2ea7bf4701 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:01.123403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:01.124582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:01.125582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:01.126017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] PATH_INFO: `/v3/users/dda3f141c850493885c86dec06006c27` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:01.127472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dda3f141c850493885c86dec06006c27)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:01.129953 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:01.378405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dda3f141c850493885c86dec06006c27'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.395303 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dda3f141c850493885c86dec06006c27'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.396538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dda3f141c850493885c86dec06006c27'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.397672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dda3f141c850493885c86dec06006c27'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.477722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f4b05061-f39a-412b-8f6a-cff6f75ed782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dda3f141c850493885c86dec06006c27'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:01.502179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:01.509151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1231/2481] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:01 2022] DELETE /identity/v3/users/dda3f141c850493885c86dec06006c27 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:02.165794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:02.167295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:02.187760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dd1c3-704e-4dda-ad9e-7f709e181541 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:02.276736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:02.278860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:02.279149 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:02.279434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] PATH_INFO: `/v3/projects/a9a05980bcc34cf1a771863248627673` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:02.280728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a05980bcc34cf1a771863248627673)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:02.282698 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:02.367033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:02.387763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a05980bcc34cf1a771863248627673', 'request_id': 'req-f658a690-baca-4b52-adf1-c8ffb7863a3b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:02.483051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a05980bcc34cf1a771863248627673', 'request_id': 'req-f658a690-baca-4b52-adf1-c8ffb7863a3b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:02.489232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a05980bcc34cf1a771863248627673', 'request_id': 'req-f658a690-baca-4b52-adf1-c8ffb7863a3b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:02.509416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] The token cache is being invalidate because project a9a05980bcc34cf1a771863248627673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:02.509773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:02.510567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-f658a690-baca-4b52-adf1-c8ffb7863a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:02.518706 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa21ec-4358-4104-bed1-d53f7d0797df None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:02.524295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa21ec-4358-4104-bed1-d53f7d0797df None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:02.524295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa21ec-4358-4104-bed1-d53f7d0797df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:02.526429 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1251/2482] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:02 2022] DELETE /identity/v3/projects/a9a05980bcc34cf1a771863248627673 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:02.555423 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a8fa21ec-4358-4104-bed1-d53f7d0797df None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:02.556983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a8fa21ec-4358-4104-bed1-d53f7d0797df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:02.689895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1232/2483] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:41:02.697615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c11d9-b8c8-4ad8-acac-feb89c4b23fd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:02.703599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c11d9-b8c8-4ad8-acac-feb89c4b23fd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:02.703599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c11d9-b8c8-4ad8-acac-feb89c4b23fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:02.720076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5f4c11d9-b8c8-4ad8-acac-feb89c4b23fd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:02.721973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5f4c11d9-b8c8-4ad8-acac-feb89c4b23fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:02.733789 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1252/2484] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:02.737883 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa21ec-4358-4104-bed1-d53f7d0797df None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:02.749311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa21ec-4358-4104-bed1-d53f7d0797df None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:02.754456 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:02.755209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:02.755573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:02.755928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:02.757545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:41:02.757545 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:02.758946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:02.774730 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1233/2485] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:41:02.778826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c11d9-b8c8-4ad8-acac-feb89c4b23fd None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:02.789974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c11d9-b8c8-4ad8-acac-feb89c4b23fd None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:02.801661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2061fa-1d25-43b8-953a-8b9e5ee61113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:02.801661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2061fa-1d25-43b8-953a-8b9e5ee61113 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:02.801661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2061fa-1d25-43b8-953a-8b9e5ee61113 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:02.801661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2061fa-1d25-43b8-953a-8b9e5ee61113 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:02.802299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2061fa-1d25-43b8-953a-8b9e5ee61113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:02.802299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2061fa-1d25-43b8-953a-8b9e5ee61113 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:02.898160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1253/2486] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:02 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:41:02.902322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:02.915271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bc886-d38b-412c-8ba6-748c39c601bc admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:02.983201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276bbbd-31cf-4e0f-9a90-b0c203834efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:02.983936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276bbbd-31cf-4e0f-9a90-b0c203834efc admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:02.984361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276bbbd-31cf-4e0f-9a90-b0c203834efc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:02.984676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276bbbd-31cf-4e0f-9a90-b0c203834efc admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:02.985371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276bbbd-31cf-4e0f-9a90-b0c203834efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:02.987324 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276bbbd-31cf-4e0f-9a90-b0c203834efc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:03.154779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1234/2487] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:02 2022] POST /identity/v3/users => generated 294 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:03.158846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2061fa-1d25-43b8-953a-8b9e5ee61113 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:03.171774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2061fa-1d25-43b8-953a-8b9e5ee61113 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:03.182244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb6c64-3d7d-4e0c-b3b0-b6c31dd8e618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:03.182244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb6c64-3d7d-4e0c-b3b0-b6c31dd8e618 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:03.182244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb6c64-3d7d-4e0c-b3b0-b6c31dd8e618 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:03.182244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb6c64-3d7d-4e0c-b3b0-b6c31dd8e618 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:03.182692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb6c64-3d7d-4e0c-b3b0-b6c31dd8e618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:03.182692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb6c64-3d7d-4e0c-b3b0-b6c31dd8e618 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:03.208270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1254/2488] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:03.212285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276bbbd-31cf-4e0f-9a90-b0c203834efc admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:03.222860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276bbbd-31cf-4e0f-9a90-b0c203834efc admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:03.227954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba6b83-dbc4-408b-88b8-1ea473968c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:03.228825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba6b83-dbc4-408b-88b8-1ea473968c1c admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:03.229209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba6b83-dbc4-408b-88b8-1ea473968c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:03.229556 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba6b83-dbc4-408b-88b8-1ea473968c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:03.230355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba6b83-dbc4-408b-88b8-1ea473968c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:03.232038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba6b83-dbc4-408b-88b8-1ea473968c1c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:03.244941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1235/2489] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:03.250541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb6c64-3d7d-4e0c-b3b0-b6c31dd8e618 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:03.260610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb6c64-3d7d-4e0c-b3b0-b6c31dd8e618 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:03.265171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5960fae-7013-44be-929d-99635dc099ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:03.266649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5960fae-7013-44be-929d-99635dc099ae admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:03.266934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5960fae-7013-44be-929d-99635dc099ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:03.267211 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5960fae-7013-44be-929d-99635dc099ae admin admin] PATH_INFO: `/v3/projects/f18db1e68bf541ef8d90a2d2e6b681a6/users/18b79a95072d40ab9a8d3dbf1759324e/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:03.286441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5960fae-7013-44be-929d-99635dc099ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18db1e68bf541ef8d90a2d2e6b681a6, user_id=18b79a95072d40ab9a8d3dbf1759324e, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:03.289130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5960fae-7013-44be-929d-99635dc099ae admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:03.404702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1255/2490] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:03 2022] PUT /identity/v3/projects/f18db1e68bf541ef8d90a2d2e6b681a6/users/18b79a95072d40ab9a8d3dbf1759324e/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:03.416058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f173f0-6f23-4506-86ca-7f686474424a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:03.416379 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f173f0-6f23-4506-86ca-7f686474424a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:03.416751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f173f0-6f23-4506-86ca-7f686474424a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:03.444452 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f0f173f0-6f23-4506-86ca-7f686474424a None None] MFA Rules not processed for user `18b79a95072d40ab9a8d3dbf1759324e`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:03.445722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f0f173f0-6f23-4506-86ca-7f686474424a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:03.504567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:03.515871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1236/2491] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:03.521381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5960fae-7013-44be-929d-99635dc099ae admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:03.591368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5960fae-7013-44be-929d-99635dc099ae admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:03.600428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf07012-ba5c-4087-b5c6-253b15093d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:03.601156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07012-ba5c-4087-b5c6-253b15093d5c admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:03.601596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07012-ba5c-4087-b5c6-253b15093d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:03.601900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07012-ba5c-4087-b5c6-253b15093d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:03.602610 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf07012-ba5c-4087-b5c6-253b15093d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:03.604570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf07012-ba5c-4087-b5c6-253b15093d5c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:03.706811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1256/2492] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:03 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:03.710773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f173f0-6f23-4506-86ca-7f686474424a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:03.721589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f173f0-6f23-4506-86ca-7f686474424a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:03.776854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebabd1-a54c-4361-a590-29bc17267a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:03.777521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebabd1-a54c-4361-a590-29bc17267a7a admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:03.777810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebabd1-a54c-4361-a590-29bc17267a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:03.778092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebabd1-a54c-4361-a590-29bc17267a7a admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:03.778801 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebabd1-a54c-4361-a590-29bc17267a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:03.783805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebabd1-a54c-4361-a590-29bc17267a7a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:03.872048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1237/2493] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:03 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:03.875806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf07012-ba5c-4087-b5c6-253b15093d5c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:03.886354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf07012-ba5c-4087-b5c6-253b15093d5c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:03.891034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc696b1-ceff-4edb-95d3-8bd3c823649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:03.891691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc696b1-ceff-4edb-95d3-8bd3c823649b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:03.891981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc696b1-ceff-4edb-95d3-8bd3c823649b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:03.892260 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc696b1-ceff-4edb-95d3-8bd3c823649b admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:03.893033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc696b1-ceff-4edb-95d3-8bd3c823649b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:03.894627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc696b1-ceff-4edb-95d3-8bd3c823649b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:03.911360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1257/2494] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:03.917378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebabd1-a54c-4361-a590-29bc17267a7a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:03.948967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebabd1-a54c-4361-a590-29bc17267a7a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:03.949339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5a0f6-7fa6-409b-af1e-fc1b5a053585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:03.950308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a0f6-7fa6-409b-af1e-fc1b5a053585 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:03.950603 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a0f6-7fa6-409b-af1e-fc1b5a053585 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:03.950876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a0f6-7fa6-409b-af1e-fc1b5a053585 admin admin] PATH_INFO: `/v3/projects/8a6335095fbc48d48d94a44db3bd82df/users/5fd344d5984342d39a34d1427adce3f4/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:03.979675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5a0f6-7fa6-409b-af1e-fc1b5a053585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6335095fbc48d48d94a44db3bd82df, user_id=5fd344d5984342d39a34d1427adce3f4, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:03.979675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5a0f6-7fa6-409b-af1e-fc1b5a053585 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:04.062949 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1238/2495] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:03 2022] PUT /identity/v3/projects/8a6335095fbc48d48d94a44db3bd82df/users/5fd344d5984342d39a34d1427adce3f4/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:04.065385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc696b1-ceff-4edb-95d3-8bd3c823649b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:04.075678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc696b1-ceff-4edb-95d3-8bd3c823649b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:04.100676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:04.133686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358bdfa-4010-479e-b937-00b32105c2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:04.137642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358bdfa-4010-479e-b937-00b32105c2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:04.137642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358bdfa-4010-479e-b937-00b32105c2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:04.137642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358bdfa-4010-479e-b937-00b32105c2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:04.137642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358bdfa-4010-479e-b937-00b32105c2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:04.137642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358bdfa-4010-479e-b937-00b32105c2f5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:04.147751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1258/2496] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:04.152066 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5a0f6-7fa6-409b-af1e-fc1b5a053585 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:04.165343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5a0f6-7fa6-409b-af1e-fc1b5a053585 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:04.170743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:04.171613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:04.171905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:04.172184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] PATH_INFO: `/v3/projects/8a6335095fbc48d48d94a44db3bd82df/users/5fd344d5984342d39a34d1427adce3f4/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:04.174427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6335095fbc48d48d94a44db3bd82df, user_id=5fd344d5984342d39a34d1427adce3f4, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:04.177089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:04.188182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1239/2497] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:04 2022] PUT /identity/v3/projects/8a6335095fbc48d48d94a44db3bd82df/users/5fd344d5984342d39a34d1427adce3f4/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:04.198947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e373e-c0f5-47b3-8048-8ba36c43f756 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:04.199446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e373e-c0f5-47b3-8048-8ba36c43f756 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:04.199717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e373e-c0f5-47b3-8048-8ba36c43f756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:04.241677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-662e373e-c0f5-47b3-8048-8ba36c43f756 None None] MFA Rules not processed for user `5fd344d5984342d39a34d1427adce3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:04.241677 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-662e373e-c0f5-47b3-8048-8ba36c43f756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:04.326001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1259/2498] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:41:04.337124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:04.338887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:04.399059 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960e64-2d1f-4d63-bbb5-fcd27272a4c1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:04.402920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17111d5-51f5-4051-b109-a447dfbf5023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:04.408758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17111d5-51f5-4051-b109-a447dfbf5023 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:04.408758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17111d5-51f5-4051-b109-a447dfbf5023 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:04.408758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17111d5-51f5-4051-b109-a447dfbf5023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:04.414245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17111d5-51f5-4051-b109-a447dfbf5023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:04.415684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17111d5-51f5-4051-b109-a447dfbf5023 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:04.426296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1240/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:05.653454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e373e-c0f5-47b3-8048-8ba36c43f756 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:05.715507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e373e-c0f5-47b3-8048-8ba36c43f756 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:05.720630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:05.721507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:05.721791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:05.722068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] PATH_INFO: `/v3/users/18b79a95072d40ab9a8d3dbf1759324e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:05.723245 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b79a95072d40ab9a8d3dbf1759324e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:05.725202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:05.915655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b79a95072d40ab9a8d3dbf1759324e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:05.917640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:05.928528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b79a95072d40ab9a8d3dbf1759324e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:05.935714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b79a95072d40ab9a8d3dbf1759324e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:05.938097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b79a95072d40ab9a8d3dbf1759324e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:05.938097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b79a95072d40ab9a8d3dbf1759324e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:06.049082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1260/2500] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:05 2022] DELETE /identity/v3/users/18b79a95072d40ab9a8d3dbf1759324e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:06.053521 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2fc6d-fa45-41bf-8b33-475edc1a88e1 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:06.054020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2fc6d-fa45-41bf-8b33-475edc1a88e1 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:06.054131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2fc6d-fa45-41bf-8b33-475edc1a88e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:06.072056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c3a2fc6d-fa45-41bf-8b33-475edc1a88e1 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:06.073312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c3a2fc6d-fa45-41bf-8b33-475edc1a88e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:06.117186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:06.133469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1241/2501] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:06.143847 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:06.145363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:06.211299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bb1a-4f1a-4fb5-9f88-767cdc0364bd admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:06.214951 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd069da7-192e-43a6-ab3d-f90d3630f4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:06.215637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd069da7-192e-43a6-ab3d-f90d3630f4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:06.215927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd069da7-192e-43a6-ab3d-f90d3630f4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:06.216210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd069da7-192e-43a6-ab3d-f90d3630f4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:06.225667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd069da7-192e-43a6-ab3d-f90d3630f4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:06.227106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd069da7-192e-43a6-ab3d-f90d3630f4a1 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:06.236624 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1261/2502] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:06.650860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2fc6d-fa45-41bf-8b33-475edc1a88e1 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:06.653111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c3a2fc6d-fa45-41bf-8b33-475edc1a88e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:06.667394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2fc6d-fa45-41bf-8b33-475edc1a88e1 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:06.672905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:06.674021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:06.675105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:06.675105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] PATH_INFO: `/v3/projects/f18db1e68bf541ef8d90a2d2e6b681a6` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:06.676450 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18db1e68bf541ef8d90a2d2e6b681a6)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:06.678553 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:06.792948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18db1e68bf541ef8d90a2d2e6b681a6', 'request_id': 'req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:06.854589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18db1e68bf541ef8d90a2d2e6b681a6', 'request_id': 'req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:06.860637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18db1e68bf541ef8d90a2d2e6b681a6', 'request_id': 'req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:06.871582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] The token cache is being invalidate because project f18db1e68bf541ef8d90a2d2e6b681a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:06.871869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:06.872516 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:06.874079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1242/2503] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:06 2022] DELETE /identity/v3/projects/f18db1e68bf541ef8d90a2d2e6b681a6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:06.878631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd069da7-192e-43a6-ab3d-f90d3630f4a1 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:06.888389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd069da7-192e-43a6-ab3d-f90d3630f4a1 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:06.945182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:06.946010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:06.946322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:06.946602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] PATH_INFO: `/v3/users/5fd344d5984342d39a34d1427adce3f4` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:06.947943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd344d5984342d39a34d1427adce3f4)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:06.950041 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:07.126663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd344d5984342d39a34d1427adce3f4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.138828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd344d5984342d39a34d1427adce3f4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.144074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:07.146036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd344d5984342d39a34d1427adce3f4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.146702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd344d5984342d39a34d1427adce3f4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.147310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7a69127d-0ec5-45d6-bc12-c117b1192f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd344d5984342d39a34d1427adce3f4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.258264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1262/2504] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:06 2022] DELETE /identity/v3/users/5fd344d5984342d39a34d1427adce3f4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:07.643076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:07.644577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:07.658612 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb378c-c62a-4b1d-9e1d-3b3125142cf9 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:07.729203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:07.730082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:07.730367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:07.730647 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] PATH_INFO: `/v3/projects/8a6335095fbc48d48d94a44db3bd82df` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:07.731839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6335095fbc48d48d94a44db3bd82df)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:07.733983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:07.877860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6335095fbc48d48d94a44db3bd82df', 'request_id': 'req-437b1f62-100a-465a-ad92-ad304a5c4737'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.963784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6335095fbc48d48d94a44db3bd82df', 'request_id': 'req-437b1f62-100a-465a-ad92-ad304a5c4737'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.973663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6335095fbc48d48d94a44db3bd82df', 'request_id': 'req-437b1f62-100a-465a-ad92-ad304a5c4737'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.997873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] The token cache is being invalidate because project 8a6335095fbc48d48d94a44db3bd82df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:07.998176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:07.998839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-437b1f62-100a-465a-ad92-ad304a5c4737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:08.004559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1243/2505] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:07 2022] DELETE /identity/v3/projects/8a6335095fbc48d48d94a44db3bd82df => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:08.005526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7e570-e0e0-45f6-8281-d3e7666a801d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.005984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7e570-e0e0-45f6-8281-d3e7666a801d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.006082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7e570-e0e0-45f6-8281-d3e7666a801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.028594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6ba7e570-e0e0-45f6-8281-d3e7666a801d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:08.029824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6ba7e570-e0e0-45f6-8281-d3e7666a801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:08.112272 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1263/2506] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:08.121933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e824ce-392b-4cf9-a112-a806cfe6b83d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.122228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e824ce-392b-4cf9-a112-a806cfe6b83d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.122535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e824ce-392b-4cf9-a112-a806cfe6b83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.129369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:08.143733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-97e824ce-392b-4cf9-a112-a806cfe6b83d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:08.145081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-97e824ce-392b-4cf9-a112-a806cfe6b83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:08.154388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1244/2507] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:41:08.159692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7e570-e0e0-45f6-8281-d3e7666a801d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:08.170990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7e570-e0e0-45f6-8281-d3e7666a801d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:08.176509 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:08.177579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.178056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.178411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.179389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:41:08.179777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:08.181891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:08.196913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1264/2508] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:41:08.201570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e824ce-392b-4cf9-a112-a806cfe6b83d None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:08.212103 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e824ce-392b-4cf9-a112-a806cfe6b83d None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:08.220208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b0637-1b57-4384-936f-3aab3ee7c021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:08.221926 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b0637-1b57-4384-936f-3aab3ee7c021 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.222495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b0637-1b57-4384-936f-3aab3ee7c021 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.230609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b0637-1b57-4384-936f-3aab3ee7c021 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.230609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b0637-1b57-4384-936f-3aab3ee7c021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:08.230609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b0637-1b57-4384-936f-3aab3ee7c021 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:08.371178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1245/2509] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:08 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:08.375441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:08.385691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bac3e9-7d07-4d1a-8e74-e5d2999505ab admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:08.442310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491268f2-6aaa-4f2e-b0b7-fcec7a302999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:08.442982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491268f2-6aaa-4f2e-b0b7-fcec7a302999 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.443292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491268f2-6aaa-4f2e-b0b7-fcec7a302999 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.443533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491268f2-6aaa-4f2e-b0b7-fcec7a302999 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.444486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491268f2-6aaa-4f2e-b0b7-fcec7a302999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:08.446446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491268f2-6aaa-4f2e-b0b7-fcec7a302999 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:08.624160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1265/2510] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:08 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:08.626043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b0637-1b57-4384-936f-3aab3ee7c021 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:08.636839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b0637-1b57-4384-936f-3aab3ee7c021 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:08.642438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b718bb-025c-4a15-9685-b1ba43d6bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:08.643307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b718bb-025c-4a15-9685-b1ba43d6bd5b admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.643858 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b718bb-025c-4a15-9685-b1ba43d6bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.643940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b718bb-025c-4a15-9685-b1ba43d6bd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.644669 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b718bb-025c-4a15-9685-b1ba43d6bd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:08.646301 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b718bb-025c-4a15-9685-b1ba43d6bd5b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:08.657641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1246/2511] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:08.661704 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491268f2-6aaa-4f2e-b0b7-fcec7a302999 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:08.672918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491268f2-6aaa-4f2e-b0b7-fcec7a302999 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:08.677780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e0067-bc56-4b38-b8da-91312c76f383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:08.678446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0067-bc56-4b38-b8da-91312c76f383 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.678755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0067-bc56-4b38-b8da-91312c76f383 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.679037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0067-bc56-4b38-b8da-91312c76f383 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.679767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0067-bc56-4b38-b8da-91312c76f383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:08.681404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0067-bc56-4b38-b8da-91312c76f383 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:08.693246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1266/2512] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:41:08.698270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b718bb-025c-4a15-9685-b1ba43d6bd5b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:08.709490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b718bb-025c-4a15-9685-b1ba43d6bd5b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:08.714230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:08.715184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.715506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.715750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] PATH_INFO: `/v3/projects/717e38e9bd244b4b93f91c8f244841bf/users/bfa02729930541659711f6abbb596d49/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.732206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=717e38e9bd244b4b93f91c8f244841bf, user_id=bfa02729930541659711f6abbb596d49, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:08.735000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:08.820027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1247/2513] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:08 2022] PUT /identity/v3/projects/717e38e9bd244b4b93f91c8f244841bf/users/bfa02729930541659711f6abbb596d49/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:08.832320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d716bd-1ae4-4700-a4aa-5a5a3cc8d0cd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:08.832507 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d716bd-1ae4-4700-a4aa-5a5a3cc8d0cd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:08.832824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d716bd-1ae4-4700-a4aa-5a5a3cc8d0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:08.852999 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-43d716bd-1ae4-4700-a4aa-5a5a3cc8d0cd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:08.854203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-43d716bd-1ae4-4700-a4aa-5a5a3cc8d0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:08.923224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:08.935626 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1267/2514] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:08.946314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:08.947836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:09.027439 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa94a1-d994-4a8f-b6d9-fffd8f39bd17 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:09.031250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6065b97-2ab4-4beb-8068-408936425dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:09.031885 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6065b97-2ab4-4beb-8068-408936425dec service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:09.032167 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6065b97-2ab4-4beb-8068-408936425dec service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:09.032444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6065b97-2ab4-4beb-8068-408936425dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:09.041975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6065b97-2ab4-4beb-8068-408936425dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:09.043403 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6065b97-2ab4-4beb-8068-408936425dec service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:09.053280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1248/2515] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:10.405157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87099a-318c-41ef-80ea-3eea58eaa7b4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:10.405708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87099a-318c-41ef-80ea-3eea58eaa7b4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:10.405708 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87099a-318c-41ef-80ea-3eea58eaa7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:10.435449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-9b87099a-318c-41ef-80ea-3eea58eaa7b4 None None] MFA Rules not processed for user `bfa02729930541659711f6abbb596d49`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:10.436893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9b87099a-318c-41ef-80ea-3eea58eaa7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:10.526943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1268/2516] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:10.528114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6065b97-2ab4-4beb-8068-408936425dec service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:10.539180 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6065b97-2ab4-4beb-8068-408936425dec service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:10.552792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cd86c-4000-49c6-9c1e-500e61e4f8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:10.553585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cd86c-4000-49c6-9c1e-500e61e4f8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:10.553984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cd86c-4000-49c6-9c1e-500e61e4f8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:10.554334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cd86c-4000-49c6-9c1e-500e61e4f8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:10.555151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cd86c-4000-49c6-9c1e-500e61e4f8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:10.557114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cd86c-4000-49c6-9c1e-500e61e4f8d9 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:10.658080 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1249/2517] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:10 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:41:10.662046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87099a-318c-41ef-80ea-3eea58eaa7b4 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:10.676795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87099a-318c-41ef-80ea-3eea58eaa7b4 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:10.739006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a90f6-ac8e-4b8f-abc0-107facc57e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:10.739700 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a90f6-ac8e-4b8f-abc0-107facc57e0b admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:10.739996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a90f6-ac8e-4b8f-abc0-107facc57e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:10.740262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a90f6-ac8e-4b8f-abc0-107facc57e0b admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:10.740995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a90f6-ac8e-4b8f-abc0-107facc57e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:10.742914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a90f6-ac8e-4b8f-abc0-107facc57e0b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:10.849005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1269/2518] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:10 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:10.852702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cd86c-4000-49c6-9c1e-500e61e4f8d9 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:10.863566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cd86c-4000-49c6-9c1e-500e61e4f8d9 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:10.868174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f57c-c595-4b04-8e70-365054323545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:10.868875 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f57c-c595-4b04-8e70-365054323545 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:10.869166 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f57c-c595-4b04-8e70-365054323545 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:10.869451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f57c-c595-4b04-8e70-365054323545 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:10.870216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397f57c-c595-4b04-8e70-365054323545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:10.876146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397f57c-c595-4b04-8e70-365054323545 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:10.887410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1250/2519] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:10.891865 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a90f6-ac8e-4b8f-abc0-107facc57e0b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:10.905296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a90f6-ac8e-4b8f-abc0-107facc57e0b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:10.912244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c89a42-f3cb-4301-88fc-97d04367c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:10.913367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c89a42-f3cb-4301-88fc-97d04367c5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:10.913735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c89a42-f3cb-4301-88fc-97d04367c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:10.913922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c89a42-f3cb-4301-88fc-97d04367c5bd admin admin] PATH_INFO: `/v3/projects/454ff6f9a1e84bfbbc2d65ae45f9666b/users/3601373c096b47f985bdc0cd3e77078d/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:10.932772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c89a42-f3cb-4301-88fc-97d04367c5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=454ff6f9a1e84bfbbc2d65ae45f9666b, user_id=3601373c096b47f985bdc0cd3e77078d, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:10.939130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c89a42-f3cb-4301-88fc-97d04367c5bd admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:11.049717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1270/2520] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:10 2022] PUT /identity/v3/projects/454ff6f9a1e84bfbbc2d65ae45f9666b/users/3601373c096b47f985bdc0cd3e77078d/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:11.053884 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f57c-c595-4b04-8e70-365054323545 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:11.064105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f57c-c595-4b04-8e70-365054323545 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:11.125123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224f205-9be5-456b-8884-6d4abe080f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:11.125783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224f205-9be5-456b-8884-6d4abe080f3e admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:11.126071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224f205-9be5-456b-8884-6d4abe080f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:11.126469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224f205-9be5-456b-8884-6d4abe080f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:11.127207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224f205-9be5-456b-8884-6d4abe080f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:11.128857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224f205-9be5-456b-8884-6d4abe080f3e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:11.139386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1251/2521] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:11.144061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c89a42-f3cb-4301-88fc-97d04367c5bd admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:11.154844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c89a42-f3cb-4301-88fc-97d04367c5bd admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:11.160064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92aed88-0a20-4941-bce7-9dbe307a3f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:11.160982 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aed88-0a20-4941-bce7-9dbe307a3f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:11.161273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aed88-0a20-4941-bce7-9dbe307a3f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:11.161609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aed88-0a20-4941-bce7-9dbe307a3f3b admin admin] PATH_INFO: `/v3/projects/454ff6f9a1e84bfbbc2d65ae45f9666b/users/3601373c096b47f985bdc0cd3e77078d/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:11.163597 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92aed88-0a20-4941-bce7-9dbe307a3f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=454ff6f9a1e84bfbbc2d65ae45f9666b, user_id=3601373c096b47f985bdc0cd3e77078d, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:11.166274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92aed88-0a20-4941-bce7-9dbe307a3f3b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:11.178040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1271/2522] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:11 2022] PUT /identity/v3/projects/454ff6f9a1e84bfbbc2d65ae45f9666b/users/3601373c096b47f985bdc0cd3e77078d/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:11.415667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224f205-9be5-456b-8884-6d4abe080f3e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:11.421681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:11.472973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224f205-9be5-456b-8884-6d4abe080f3e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:11.543791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:11.544502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:11.545081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:11.545435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:11.548136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:11.558085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:11.559547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:11.568843 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1252/2523] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:12.722178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa571-c70e-4481-aa62-57bd8ccadb3e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:12.722727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa571-c70e-4481-aa62-57bd8ccadb3e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:12.722835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa571-c70e-4481-aa62-57bd8ccadb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:12.750852 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6c3aa571-c70e-4481-aa62-57bd8ccadb3e None None] MFA Rules not processed for user `3601373c096b47f985bdc0cd3e77078d`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:12.752079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6c3aa571-c70e-4481-aa62-57bd8ccadb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:12.780306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:12.821945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1272/2524] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:12.833771 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:12.835192 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:12.895177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa901d-d3b1-4c90-a629-bd8295ab821f service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:12.900185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcda0c-eea7-4bca-9e72-77c6526add03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:12.900185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcda0c-eea7-4bca-9e72-77c6526add03 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:12.900185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcda0c-eea7-4bca-9e72-77c6526add03 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:12.900185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcda0c-eea7-4bca-9e72-77c6526add03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:12.968058 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcda0c-eea7-4bca-9e72-77c6526add03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:12.969819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcda0c-eea7-4bca-9e72-77c6526add03 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:12.979055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1253/2525] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:14.737576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3aa571-c70e-4481-aa62-57bd8ccadb3e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:14.742151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6c3aa571-c70e-4481-aa62-57bd8ccadb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:14.752729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3aa571-c70e-4481-aa62-57bd8ccadb3e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:14.756358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:14.757006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:14.757317 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:14.757621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:14.766877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:14.768325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:14.778527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1273/2526] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:15.300282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f50a0c-f8ac-4f8a-9e37-d49f16648b6c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:15.300282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f50a0c-f8ac-4f8a-9e37-d49f16648b6c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:15.300282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f50a0c-f8ac-4f8a-9e37-d49f16648b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:15.316369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-e4f50a0c-f8ac-4f8a-9e37-d49f16648b6c None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:15.317974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e4f50a0c-f8ac-4f8a-9e37-d49f16648b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:15.330795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:15.376975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1254/2527] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:41:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:15.389512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:15.391445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:15.471882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79cfbf-ce80-4eab-a315-0ad42a5ead4d service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:15.477976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:15.483126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:15.483126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:15.483126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:15.488726 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:15.490345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:15.499172 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1274/2528] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:41:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:41:15.596162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f50a0c-f8ac-4f8a-9e37-d49f16648b6c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:15.606907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f50a0c-f8ac-4f8a-9e37-d49f16648b6c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:15.610525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:15.611189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:15.611475 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:15.611754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:15.613698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:15.718398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:15.720413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:15.787001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1255/2529] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:15.829953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:15.835312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:15.868032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbad277-4ec2-40f7-9ec5-d2406216214f service swift] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:15.872725 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:15.876986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:15.876986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:15.876986 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:15.877548 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:15.990874 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:15.992249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:16.005443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1275/2530] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:23.381050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:23.382558 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:23.414039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c81ca1-c20f-42e4-9935-85336c0130bc service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:23.418299 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:23.419319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:23.419865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:23.420371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] PATH_INFO: `/v3/users/052ae1db33014bb5915dc228f1b76d5c` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:23.421902 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=052ae1db33014bb5915dc228f1b76d5c)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:23.424073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:23.667117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052ae1db33014bb5915dc228f1b76d5c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:23.699418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052ae1db33014bb5915dc228f1b76d5c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:23.700908 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052ae1db33014bb5915dc228f1b76d5c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:23.701540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052ae1db33014bb5915dc228f1b76d5c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:23.818154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-30d8eb69-41c7-492c-aa24-cdc58953af05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052ae1db33014bb5915dc228f1b76d5c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:23.855579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1256/2531] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:23 2022] DELETE /identity/v3/users/052ae1db33014bb5915dc228f1b76d5c => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:24.337015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:24.350043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4152fe-7a4a-46c2-a794-37c92b3de6a7 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:24.386911 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:24.414262 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:24.415044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:24.415273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:24.415551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] PATH_INFO: `/v3/projects/45031128a55e42989da61e01948538d3` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:24.416862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45031128a55e42989da61e01948538d3)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:24.419353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:24.606940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45031128a55e42989da61e01948538d3', 'request_id': 'req-687857b2-3e71-42d1-b6b1-c514ed8c54d4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:24.672613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45031128a55e42989da61e01948538d3', 'request_id': 'req-687857b2-3e71-42d1-b6b1-c514ed8c54d4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:24.688344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45031128a55e42989da61e01948538d3', 'request_id': 'req-687857b2-3e71-42d1-b6b1-c514ed8c54d4'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:24.700988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] The token cache is being invalidate because project 45031128a55e42989da61e01948538d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:24.701214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:24.701712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-687857b2-3e71-42d1-b6b1-c514ed8c54d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:24.704118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1276/2532] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:24 2022] DELETE /identity/v3/projects/45031128a55e42989da61e01948538d3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:24.710544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77c958-7c04-4216-88e3-a9718f9e3b77 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:24.711544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77c958-7c04-4216-88e3-a9718f9e3b77 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:24.712033 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77c958-7c04-4216-88e3-a9718f9e3b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:24.749172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-1f77c958-7c04-4216-88e3-a9718f9e3b77 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:24.751803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1f77c958-7c04-4216-88e3-a9718f9e3b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:24.784393 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:24.831151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1257/2533] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:24.839443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bae56-0374-498f-9112-8e2570525d60 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:24.839443 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bae56-0374-498f-9112-8e2570525d60 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:24.839606 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bae56-0374-498f-9112-8e2570525d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:24.860106 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a14bae56-0374-498f-9112-8e2570525d60 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:24.861512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a14bae56-0374-498f-9112-8e2570525d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:24.869831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1277/2534] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:24.880061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77c958-7c04-4216-88e3-a9718f9e3b77 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:24.889945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77c958-7c04-4216-88e3-a9718f9e3b77 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:24.895176 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:24.895966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:24.896255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:24.896681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:24.897408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:41:24.897716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:24.899462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:24.911644 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1258/2535] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:41:24.915987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bae56-0374-498f-9112-8e2570525d60 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:24.925343 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bae56-0374-498f-9112-8e2570525d60 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:24.929966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707c1c-4176-454f-a618-c7708919aabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:24.930618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707c1c-4176-454f-a618-c7708919aabd admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:24.930900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707c1c-4176-454f-a618-c7708919aabd admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:24.931188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707c1c-4176-454f-a618-c7708919aabd admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:24.931877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74707c1c-4176-454f-a618-c7708919aabd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:24.933816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74707c1c-4176-454f-a618-c7708919aabd admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:25.013855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1278/2536] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:24 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:25.015355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:25.026481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36858e5-1fc4-4a84-ac20-30e07b986063 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:25.084459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772561-d24a-40a4-9650-ef4bad6b6b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:25.085181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772561-d24a-40a4-9650-ef4bad6b6b8b admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:25.085472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772561-d24a-40a4-9650-ef4bad6b6b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:25.085824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772561-d24a-40a4-9650-ef4bad6b6b8b admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:25.086523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd772561-d24a-40a4-9650-ef4bad6b6b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:25.088432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd772561-d24a-40a4-9650-ef4bad6b6b8b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:25.186121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707c1c-4176-454f-a618-c7708919aabd admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:25.186380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1259/2537] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:25 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:25.199296 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707c1c-4176-454f-a618-c7708919aabd admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:25.204898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d77ca-333e-4573-8fcc-2834e9ae032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:25.205850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d77ca-333e-4573-8fcc-2834e9ae032d admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:25.206246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d77ca-333e-4573-8fcc-2834e9ae032d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:25.206630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d77ca-333e-4573-8fcc-2834e9ae032d admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:25.207483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d77ca-333e-4573-8fcc-2834e9ae032d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:25.209524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d77ca-333e-4573-8fcc-2834e9ae032d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:25.225551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1279/2538] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:25.227181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772561-d24a-40a4-9650-ef4bad6b6b8b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:25.238903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772561-d24a-40a4-9650-ef4bad6b6b8b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:25.243916 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98d8c5-a7e7-46a0-baad-e1a2bf21aa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:25.244860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d8c5-a7e7-46a0-baad-e1a2bf21aa9d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:25.244969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d8c5-a7e7-46a0-baad-e1a2bf21aa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:25.245221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d8c5-a7e7-46a0-baad-e1a2bf21aa9d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:25.245967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98d8c5-a7e7-46a0-baad-e1a2bf21aa9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:25.247607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98d8c5-a7e7-46a0-baad-e1a2bf21aa9d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:25.259808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1260/2539] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:25.264210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d77ca-333e-4573-8fcc-2834e9ae032d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:25.280314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d77ca-333e-4573-8fcc-2834e9ae032d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:25.286036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:25.287409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:25.287409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:25.287845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] PATH_INFO: `/v3/projects/61d5e2a5218d47aa9a92ce3716790461/users/02f1cd3cfa784328a0cf174040878e6c/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:25.304632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d5e2a5218d47aa9a92ce3716790461, user_id=02f1cd3cfa784328a0cf174040878e6c, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:25.307349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:25.420731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1280/2540] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:25 2022] PUT /identity/v3/projects/61d5e2a5218d47aa9a92ce3716790461/users/02f1cd3cfa784328a0cf174040878e6c/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:25.430905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7a37-4fe8-4562-877e-e5f73bcc9770 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:25.431207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7a37-4fe8-4562-877e-e5f73bcc9770 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:25.435038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7a37-4fe8-4562-877e-e5f73bcc9770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:25.458057 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-67bd7a37-4fe8-4562-877e-e5f73bcc9770 None None] MFA Rules not processed for user `02f1cd3cfa784328a0cf174040878e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:25.459273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-67bd7a37-4fe8-4562-877e-e5f73bcc9770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:25.525142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1261/2541] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:25.536790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:25.538634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:25.610168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:25.626778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e9544-91ab-4b1e-8fcb-124be607fdd4 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:25.632803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2890607-cbb9-4166-a717-8abd93d13454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:25.633459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2890607-cbb9-4166-a717-8abd93d13454 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:25.633747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2890607-cbb9-4166-a717-8abd93d13454 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:25.634055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2890607-cbb9-4166-a717-8abd93d13454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:25.643037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2890607-cbb9-4166-a717-8abd93d13454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:25.644689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2890607-cbb9-4166-a717-8abd93d13454 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:25.653808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1281/2542] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:28.896834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd7a37-4fe8-4562-877e-e5f73bcc9770 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:28.927874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:28.964092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd7a37-4fe8-4562-877e-e5f73bcc9770 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:28.969419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:28.970287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:28.970607 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:28.970887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] PATH_INFO: `/v3/users/02f1cd3cfa784328a0cf174040878e6c` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:28.972088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=02f1cd3cfa784328a0cf174040878e6c)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:28.974052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:29.217586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02f1cd3cfa784328a0cf174040878e6c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:29.225373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02f1cd3cfa784328a0cf174040878e6c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:29.226433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02f1cd3cfa784328a0cf174040878e6c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:29.227055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02f1cd3cfa784328a0cf174040878e6c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:29.326307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02f1cd3cfa784328a0cf174040878e6c'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:29.345541 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1262/2543] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:28 2022] DELETE /identity/v3/users/02f1cd3cfa784328a0cf174040878e6c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:29.350199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0718d-69b1-4beb-b768-38dda0f1e3f2 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:29.350744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0718d-69b1-4beb-b768-38dda0f1e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:29.350891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0718d-69b1-4beb-b768-38dda0f1e3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:29.368481 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-e5a0718d-69b1-4beb-b768-38dda0f1e3f2 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:29.369704 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e5a0718d-69b1-4beb-b768-38dda0f1e3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:29.441925 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1282/2544] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:41:29.454770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:29.456759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:29.526419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc727a57-df8e-46bf-84b6-f2a85be80f4f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:29.531565 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbbfd0-9930-419d-bf82-52decddc06f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:29.532649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbfd0-9930-419d-bf82-52decddc06f0 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:29.532936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbfd0-9930-419d-bf82-52decddc06f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:29.533242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbfd0-9930-419d-bf82-52decddc06f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:29.544294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbbfd0-9930-419d-bf82-52decddc06f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:29.545767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbbfd0-9930-419d-bf82-52decddc06f0 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:29.563236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1263/2545] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:30.055685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0718d-69b1-4beb-b768-38dda0f1e3f2 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:30.057144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-e5a0718d-69b1-4beb-b768-38dda0f1e3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:30.070802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0718d-69b1-4beb-b768-38dda0f1e3f2 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:30.075127 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:30.076014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.076302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.076616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] PATH_INFO: `/v3/projects/61d5e2a5218d47aa9a92ce3716790461` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.081791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d5e2a5218d47aa9a92ce3716790461)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:30.081791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:30.215461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d5e2a5218d47aa9a92ce3716790461', 'request_id': 'req-fdd7c303-3456-47fc-b450-3b69b7fa59ca'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:30.283197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d5e2a5218d47aa9a92ce3716790461', 'request_id': 'req-fdd7c303-3456-47fc-b450-3b69b7fa59ca'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:30.289066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d5e2a5218d47aa9a92ce3716790461', 'request_id': 'req-fdd7c303-3456-47fc-b450-3b69b7fa59ca'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:30.300231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] The token cache is being invalidate because project 61d5e2a5218d47aa9a92ce3716790461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:30.300558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:30.301189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fdd7c303-3456-47fc-b450-3b69b7fa59ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:30.302716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1283/2546] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:30 2022] DELETE /identity/v3/projects/61d5e2a5218d47aa9a92ce3716790461 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:30.307602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd450-35c0-47e0-9049-8959202c62f9 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.308070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd450-35c0-47e0-9049-8959202c62f9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.308186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd450-35c0-47e0-9049-8959202c62f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.326333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-463fd450-35c0-47e0-9049-8959202c62f9 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:30.327510 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-463fd450-35c0-47e0-9049-8959202c62f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:30.369203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:30.388078 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1264/2547] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:30.395842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6aa51-3d66-4dc6-9d04-6eaba7871e9d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.396305 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6aa51-3d66-4dc6-9d04-6eaba7871e9d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.396405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6aa51-3d66-4dc6-9d04-6eaba7871e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.414322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b5a6aa51-3d66-4dc6-9d04-6eaba7871e9d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:30.415497 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b5a6aa51-3d66-4dc6-9d04-6eaba7871e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:30.423774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1284/2548] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:30.427896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fd450-35c0-47e0-9049-8959202c62f9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:30.438037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fd450-35c0-47e0-9049-8959202c62f9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:30.442670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:30.443329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.443621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.443905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.444919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:41:30.445233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:30.446840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:30.458536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1265/2549] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:41:30.465850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6aa51-3d66-4dc6-9d04-6eaba7871e9d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:30.472067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6aa51-3d66-4dc6-9d04-6eaba7871e9d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:30.476766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ee0e6-4c37-4d45-8a8f-04582645db46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:30.477421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee0e6-4c37-4d45-8a8f-04582645db46 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.477754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee0e6-4c37-4d45-8a8f-04582645db46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.478023 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee0e6-4c37-4d45-8a8f-04582645db46 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.478719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ee0e6-4c37-4d45-8a8f-04582645db46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:30.480649 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ee0e6-4c37-4d45-8a8f-04582645db46 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:30.599234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1285/2550] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:30 2022] POST /identity/v3/projects => generated 403 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:30.603292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:30.614104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72af2-d067-4f23-8546-b99176b5eb46 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:30.669448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834b629-50c1-4552-966a-29cf6faca2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:30.670117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834b629-50c1-4552-966a-29cf6faca2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.670408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834b629-50c1-4552-966a-29cf6faca2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.670690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834b629-50c1-4552-966a-29cf6faca2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.671378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834b629-50c1-4552-966a-29cf6faca2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:30.673829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834b629-50c1-4552-966a-29cf6faca2a6 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:30.815482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1266/2551] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:30 2022] POST /identity/v3/users => generated 302 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:30.820015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ee0e6-4c37-4d45-8a8f-04582645db46 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:30.836461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ee0e6-4c37-4d45-8a8f-04582645db46 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:30.836461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d66097-9d92-4919-b3c8-1aef90d9ec16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:30.836461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d66097-9d92-4919-b3c8-1aef90d9ec16 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.836461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d66097-9d92-4919-b3c8-1aef90d9ec16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.836870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d66097-9d92-4919-b3c8-1aef90d9ec16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.837341 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d66097-9d92-4919-b3c8-1aef90d9ec16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:30.839118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d66097-9d92-4919-b3c8-1aef90d9ec16 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:30.850042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1286/2552] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:30.854053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834b629-50c1-4552-966a-29cf6faca2a6 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:30.865219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834b629-50c1-4552-966a-29cf6faca2a6 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:30.870424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312498d2-264d-4359-926b-848d989a5372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:30.871091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312498d2-264d-4359-926b-848d989a5372 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.871374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312498d2-264d-4359-926b-848d989a5372 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.871653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312498d2-264d-4359-926b-848d989a5372 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.872499 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312498d2-264d-4359-926b-848d989a5372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:30.874221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312498d2-264d-4359-926b-848d989a5372 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:30.889773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1267/2553] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:30.891184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d66097-9d92-4919-b3c8-1aef90d9ec16 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:30.905193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d66097-9d92-4919-b3c8-1aef90d9ec16 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:30.910668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b00f1-fd4d-4d58-bc4b-ed2c4a68d683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:30.911612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b00f1-fd4d-4d58-bc4b-ed2c4a68d683 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:30.912367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b00f1-fd4d-4d58-bc4b-ed2c4a68d683 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:30.912367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b00f1-fd4d-4d58-bc4b-ed2c4a68d683 admin admin] PATH_INFO: `/v3/projects/39e60311623f47288f0a2e6bda5c7270/users/cceabc34b65b4d209e6398aecc3f1f57/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:30.934495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b00f1-fd4d-4d58-bc4b-ed2c4a68d683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e60311623f47288f0a2e6bda5c7270, user_id=cceabc34b65b4d209e6398aecc3f1f57, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:30.937232 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b00f1-fd4d-4d58-bc4b-ed2c4a68d683 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:31.032811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1287/2554] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:30 2022] PUT /identity/v3/projects/39e60311623f47288f0a2e6bda5c7270/users/cceabc34b65b4d209e6398aecc3f1f57/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:31.047853 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c00a5-dc0e-4d4c-b8a1-13fa15797d9f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.048162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c00a5-dc0e-4d4c-b8a1-13fa15797d9f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.048444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c00a5-dc0e-4d4c-b8a1-13fa15797d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.052581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b93c0-5437-454d-a93b-67598f2d56ad None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.052881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b93c0-5437-454d-a93b-67598f2d56ad None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.053165 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b93c0-5437-454d-a93b-67598f2d56ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.060572 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:31.072282 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-035b93c0-5437-454d-a93b-67598f2d56ad None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:31.073573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-035b93c0-5437-454d-a93b-67598f2d56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:31.078215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f48c00a5-dc0e-4d4c-b8a1-13fa15797d9f None None] MFA Rules not processed for user `cceabc34b65b4d209e6398aecc3f1f57`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:31.079483 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f48c00a5-dc0e-4d4c-b8a1-13fa15797d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:31.142958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1288/2555] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jun 16 00:41:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:41:31.170219 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b93c0-5437-454d-a93b-67598f2d56ad None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:31.179702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-035b93c0-5437-454d-a93b-67598f2d56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:31.179838 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1268/2556] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:31.180933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c00a5-dc0e-4d4c-b8a1-13fa15797d9f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:31.252508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:31.279220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b93c0-5437-454d-a93b-67598f2d56ad None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:31.289477 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c00a5-dc0e-4d4c-b8a1-13fa15797d9f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:31.293738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b5bd2-fa5d-4d98-b9e9-4385743bff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:31.294434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5bd2-fa5d-4d98-b9e9-4385743bff03 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.294745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5bd2-fa5d-4d98-b9e9-4385743bff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.295000 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5bd2-fa5d-4d98-b9e9-4385743bff03 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.295722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b5bd2-fa5d-4d98-b9e9-4385743bff03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:31.297797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502d39a-1233-4035-a940-e132251d2645 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:31.298217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b5bd2-fa5d-4d98-b9e9-4385743bff03 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:31.300473 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d39a-1233-4035-a940-e132251d2645 service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.301213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d39a-1233-4035-a940-e132251d2645 service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.301748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d39a-1233-4035-a940-e132251d2645 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.316990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502d39a-1233-4035-a940-e132251d2645 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:31.319050 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502d39a-1233-4035-a940-e132251d2645 service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:31.329600 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1289/2557] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:41:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:41:31.388966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1269/2558] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:31 2022] POST /identity/v3/projects => generated 403 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:41:31.393859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502d39a-1233-4035-a940-e132251d2645 service swift] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:31.409166 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502d39a-1233-4035-a940-e132251d2645 service swift] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:31.477810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7b07-18da-47fb-97d7-46d6d09428bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:31.478502 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7b07-18da-47fb-97d7-46d6d09428bc admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.478790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7b07-18da-47fb-97d7-46d6d09428bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.479073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7b07-18da-47fb-97d7-46d6d09428bc admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.479767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa7b07-18da-47fb-97d7-46d6d09428bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:31.481909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa7b07-18da-47fb-97d7-46d6d09428bc admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:31.611386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1290/2559] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:31 2022] POST /identity/v3/users => generated 308 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:31.615368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b5bd2-fa5d-4d98-b9e9-4385743bff03 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:31.627365 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b5bd2-fa5d-4d98-b9e9-4385743bff03 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:31.632071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbb181-2507-4b84-b83c-70b044f2dd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:31.632761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbb181-2507-4b84-b83c-70b044f2dd83 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.633047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbb181-2507-4b84-b83c-70b044f2dd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.633323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbb181-2507-4b84-b83c-70b044f2dd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.634048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbb181-2507-4b84-b83c-70b044f2dd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:31.635629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbb181-2507-4b84-b83c-70b044f2dd83 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:31.658186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1270/2560] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:31.663403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7b07-18da-47fb-97d7-46d6d09428bc admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:31.675377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7b07-18da-47fb-97d7-46d6d09428bc admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:31.680365 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493add3-0474-4c85-8900-97c9176a827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:31.681371 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493add3-0474-4c85-8900-97c9176a827a admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.681664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493add3-0474-4c85-8900-97c9176a827a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.681946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493add3-0474-4c85-8900-97c9176a827a admin admin] PATH_INFO: `/v3/projects/4b4789a7544c43a29e41149af2ad2c68/users/14673d6f347f4e20b4d201a28db1b67d/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.698785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493add3-0474-4c85-8900-97c9176a827a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4789a7544c43a29e41149af2ad2c68, user_id=14673d6f347f4e20b4d201a28db1b67d, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:31.702603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493add3-0474-4c85-8900-97c9176a827a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:31.798022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1291/2561] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:31 2022] PUT /identity/v3/projects/4b4789a7544c43a29e41149af2ad2c68/users/14673d6f347f4e20b4d201a28db1b67d/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:31.802186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbb181-2507-4b84-b83c-70b044f2dd83 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:31.814218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbb181-2507-4b84-b83c-70b044f2dd83 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:31.871945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a993d-210f-4cf8-9a95-e759b6042209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:31.872638 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a993d-210f-4cf8-9a95-e759b6042209 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.872928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a993d-210f-4cf8-9a95-e759b6042209 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.873210 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a993d-210f-4cf8-9a95-e759b6042209 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.873937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a993d-210f-4cf8-9a95-e759b6042209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:31.876106 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a993d-210f-4cf8-9a95-e759b6042209 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:31.885971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1271/2562] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:41:31.890379 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493add3-0474-4c85-8900-97c9176a827a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:31.899993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:31.909086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493add3-0474-4c85-8900-97c9176a827a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:31.913890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:31.917016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.917134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.917411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] PATH_INFO: `/v3/projects/4b4789a7544c43a29e41149af2ad2c68/users/14673d6f347f4e20b4d201a28db1b67d/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.919396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4789a7544c43a29e41149af2ad2c68, user_id=14673d6f347f4e20b4d201a28db1b67d, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:31.922707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:31.933514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1292/2563] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:31 2022] PUT /identity/v3/projects/4b4789a7544c43a29e41149af2ad2c68/users/14673d6f347f4e20b4d201a28db1b67d/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:41:31.944216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddd045-e481-4664-9e87-6545e4db674f None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:31.944792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddd045-e481-4664-9e87-6545e4db674f None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:31.945073 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddd045-e481-4664-9e87-6545e4db674f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:31.976475 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-2fddd045-e481-4664-9e87-6545e4db674f None None] MFA Rules not processed for user `14673d6f347f4e20b4d201a28db1b67d`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:31.978461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2fddd045-e481-4664-9e87-6545e4db674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:32.052777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1272/2564] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:32.068419 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:32.070458 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:32.140869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bdc6a-7623-4789-83c6-60975a087e0a admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:32.144717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936e6af-25bc-470f-8b77-e22b4019f1d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:32.145369 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e6af-25bc-470f-8b77-e22b4019f1d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:32.145657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e6af-25bc-470f-8b77-e22b4019f1d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:32.145935 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e6af-25bc-470f-8b77-e22b4019f1d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:32.155125 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936e6af-25bc-470f-8b77-e22b4019f1d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:32.156661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936e6af-25bc-470f-8b77-e22b4019f1d2 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:32.166193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1293/2565] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:32 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:32.205088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddd045-e481-4664-9e87-6545e4db674f None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:32.206488 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2fddd045-e481-4664-9e87-6545e4db674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:32.266536 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:32.295075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddd045-e481-4664-9e87-6545e4db674f None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:32.301583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:32.302236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:32.302519 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:32.302798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:32.304684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:32.317250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:32.318823 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:32.328289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936e6af-25bc-470f-8b77-e22b4019f1d2 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:32.329328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1273/2566] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:41:32.344856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:32.353309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:32.427672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936e6af-25bc-470f-8b77-e22b4019f1d2 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:32.433642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:32.434711 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:32.435465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:32.435931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] PATH_INFO: `/v3/users/cceabc34b65b4d209e6398aecc3f1f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:32.437787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cceabc34b65b4d209e6398aecc3f1f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:32.440146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:32.440733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0ebd2-d2ba-41df-ace4-7e276b96ef9d service placement] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:32.445973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:32.446895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:32.447333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:32.447997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] PATH_INFO: `/v3/users/bfa02729930541659711f6abbb596d49` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:32.449643 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa02729930541659711f6abbb596d49)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:32.451910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:32.714533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cceabc34b65b4d209e6398aecc3f1f57'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.727088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cceabc34b65b4d209e6398aecc3f1f57'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.736104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cceabc34b65b4d209e6398aecc3f1f57'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.736839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cceabc34b65b4d209e6398aecc3f1f57'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.737463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-50ac2953-d634-46d6-b686-497326501c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cceabc34b65b4d209e6398aecc3f1f57'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.743896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa02729930541659711f6abbb596d49'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.750963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa02729930541659711f6abbb596d49'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.751651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa02729930541659711f6abbb596d49'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.752254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa02729930541659711f6abbb596d49'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.838448 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1294/2567] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:32 2022] DELETE /identity/v3/users/cceabc34b65b4d209e6398aecc3f1f57 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:41:32.843194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690a76f-64e8-4e44-a886-13aa62fbe416 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:32.843483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690a76f-64e8-4e44-a886-13aa62fbe416 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:32.843765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690a76f-64e8-4e44-a886-13aa62fbe416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:32.861616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d690a76f-64e8-4e44-a886-13aa62fbe416 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:32.865487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d690a76f-64e8-4e44-a886-13aa62fbe416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:32.872189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:32.889120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa02729930541659711f6abbb596d49'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:32.908882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1274/2568] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:32 2022] DELETE /identity/v3/users/bfa02729930541659711f6abbb596d49 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:41:32.924931 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1295/2569] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:41:32.934984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:32.946061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:32.993345 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:32.999145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca76f13-eb2b-4a19-a8c9-8715e5f70729 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:33.003444 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ff61e-d75f-4279-95b0-28d36b0670a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:33.004102 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ff61e-d75f-4279-95b0-28d36b0670a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:33.004387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ff61e-d75f-4279-95b0-28d36b0670a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:33.004824 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ff61e-d75f-4279-95b0-28d36b0670a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:33.065290 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ff61e-d75f-4279-95b0-28d36b0670a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:33.066717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ff61e-d75f-4279-95b0-28d36b0670a6 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:33.076645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1275/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:33.165010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690a76f-64e8-4e44-a886-13aa62fbe416 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:33.166481 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d690a76f-64e8-4e44-a886-13aa62fbe416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:33.178993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690a76f-64e8-4e44-a886-13aa62fbe416 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:33.183702 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:33.184778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:33.185061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:33.185345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] PATH_INFO: `/v3/projects/717e38e9bd244b4b93f91c8f244841bf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:33.186710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=717e38e9bd244b4b93f91c8f244841bf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:33.189193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:33.320293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '717e38e9bd244b4b93f91c8f244841bf', 'request_id': 'req-7500cefa-48ce-48ac-8577-813f9861cdfc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.397840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '717e38e9bd244b4b93f91c8f244841bf', 'request_id': 'req-7500cefa-48ce-48ac-8577-813f9861cdfc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.403987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '717e38e9bd244b4b93f91c8f244841bf', 'request_id': 'req-7500cefa-48ce-48ac-8577-813f9861cdfc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.415547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] The token cache is being invalidate because project 717e38e9bd244b4b93f91c8f244841bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:33.415838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.416499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.418086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1296/2571] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:33 2022] DELETE /identity/v3/projects/717e38e9bd244b4b93f91c8f244841bf => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:33.422662 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ff61e-d75f-4279-95b0-28d36b0670a6 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:33.433172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ff61e-d75f-4279-95b0-28d36b0670a6 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:33.489775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:33.490646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:33.490929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:33.491206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] PATH_INFO: `/v3/users/3601373c096b47f985bdc0cd3e77078d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:33.492396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3601373c096b47f985bdc0cd3e77078d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:33.494484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:33.524618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:33.526080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:33.539763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500cefa-48ce-48ac-8577-813f9861cdfc admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:33.544729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:33.550202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:33.550202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:33.550202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] PATH_INFO: `/v3/projects/39e60311623f47288f0a2e6bda5c7270` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:33.550202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e60311623f47288f0a2e6bda5c7270)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:33.550202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:33.689380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3601373c096b47f985bdc0cd3e77078d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.690479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e60311623f47288f0a2e6bda5c7270', 'request_id': 'req-2cf8972f-cce5-4f5c-a6ee-207e860cc698'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.697790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3601373c096b47f985bdc0cd3e77078d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.698513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3601373c096b47f985bdc0cd3e77078d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.699151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3601373c096b47f985bdc0cd3e77078d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.769633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e60311623f47288f0a2e6bda5c7270', 'request_id': 'req-2cf8972f-cce5-4f5c-a6ee-207e860cc698'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.775868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e60311623f47288f0a2e6bda5c7270', 'request_id': 'req-2cf8972f-cce5-4f5c-a6ee-207e860cc698'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.788032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] The token cache is being invalidate because project 39e60311623f47288f0a2e6bda5c7270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:33.788321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.789002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.790537 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1297/2572] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:33 2022] DELETE /identity/v3/projects/39e60311623f47288f0a2e6bda5c7270 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:41:33.794263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:33.795538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:33.817663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:33.831841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3601373c096b47f985bdc0cd3e77078d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:33.850965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1276/2573] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:33 2022] DELETE /identity/v3/users/3601373c096b47f985bdc0cd3e77078d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:33.861530 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8972f-cce5-4f5c-a6ee-207e860cc698 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:33.916818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:33.917701 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:33.917985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:33.918263 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] PATH_INFO: `/v3/users/14673d6f347f4e20b4d201a28db1b67d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:33.919475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14673d6f347f4e20b4d201a28db1b67d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:33.921487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.100742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:34.102295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:34.117882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b2a0-ff37-4c87-b549-989ba9d0ba9b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:34.123099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:34.123974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.124263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.124581 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] PATH_INFO: `/v3/projects/454ff6f9a1e84bfbbc2d65ae45f9666b` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.126635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=454ff6f9a1e84bfbbc2d65ae45f9666b)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:34.128645 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.135679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14673d6f347f4e20b4d201a28db1b67d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.140182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:34.148786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14673d6f347f4e20b4d201a28db1b67d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.155922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14673d6f347f4e20b4d201a28db1b67d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.157392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14673d6f347f4e20b4d201a28db1b67d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.158002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-c19ddc34-f0c3-44cb-bc89-95ca97b1718d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14673d6f347f4e20b4d201a28db1b67d'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.223973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1298/2574] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:33 2022] DELETE /identity/v3/users/14673d6f347f4e20b4d201a28db1b67d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:41:34.237076 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454ff6f9a1e84bfbbc2d65ae45f9666b', 'request_id': 'req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.312055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454ff6f9a1e84bfbbc2d65ae45f9666b', 'request_id': 'req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.318158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454ff6f9a1e84bfbbc2d65ae45f9666b', 'request_id': 'req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.330300 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] The token cache is being invalidate because project 454ff6f9a1e84bfbbc2d65ae45f9666b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:34.330580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.331233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6ceeea3b-fcf9-423d-aa1b-3ab1f4adb4d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.333199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1277/2575] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:34 2022] DELETE /identity/v3/projects/454ff6f9a1e84bfbbc2d65ae45f9666b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:41:34.337997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8e4f6-1436-4b6a-a136-c6795fbcbf1e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.338322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8e4f6-1436-4b6a-a136-c6795fbcbf1e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.338599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8e4f6-1436-4b6a-a136-c6795fbcbf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.357156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-23d8e4f6-1436-4b6a-a136-c6795fbcbf1e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:34.358364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-23d8e4f6-1436-4b6a-a136-c6795fbcbf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:34.399744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:34.420096 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1299/2576] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:34.428852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e9ed2-e024-481b-b624-05f9cd13201e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.429215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e9ed2-e024-481b-b624-05f9cd13201e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.429568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e9ed2-e024-481b-b624-05f9cd13201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.453869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-ad9e9ed2-e024-481b-b624-05f9cd13201e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:34.455108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ad9e9ed2-e024-481b-b624-05f9cd13201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:34.464865 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1278/2577] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:34.469809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8e4f6-1436-4b6a-a136-c6795fbcbf1e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:34.480101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8e4f6-1436-4b6a-a136-c6795fbcbf1e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:34.484762 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:34.485421 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.485706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.485985 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.486715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:41:34.487020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:34.488692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.501360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1300/2578] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:41:34.506473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e9ed2-e024-481b-b624-05f9cd13201e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:34.518725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e9ed2-e024-481b-b624-05f9cd13201e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:34.523779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:34.524441 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.524761 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.525041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.525750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:34.527653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.655119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1279/2579] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:34 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:41:34.659455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:34.669857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c574f8-1269-40d3-b983-34355d72c4b9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:34.697869 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:34.699783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:34.733654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f763d-f475-48fe-af88-43b12aee0fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:34.734946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f763d-f475-48fe-af88-43b12aee0fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.735355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f763d-f475-48fe-af88-43b12aee0fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.735584 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f763d-f475-48fe-af88-43b12aee0fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.736830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f763d-f475-48fe-af88-43b12aee0fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:34.739621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f763d-f475-48fe-af88-43b12aee0fd6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.776369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e322b6-7523-429d-afa1-59d3e5a8972e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:34.781074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:34.781963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.782247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.782524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] PATH_INFO: `/v3/projects/4b4789a7544c43a29e41149af2ad2c68` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.783839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b4789a7544c43a29e41149af2ad2c68)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:34.785813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.838139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1301/2580] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:34 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:34.842101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f763d-f475-48fe-af88-43b12aee0fd6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:34.851778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f763d-f475-48fe-af88-43b12aee0fd6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:34.856461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f03c6c-59de-4b5e-b5fa-e0389ebd24fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:34.858806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03c6c-59de-4b5e-b5fa-e0389ebd24fe admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.858806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03c6c-59de-4b5e-b5fa-e0389ebd24fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.858806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03c6c-59de-4b5e-b5fa-e0389ebd24fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.858806 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f03c6c-59de-4b5e-b5fa-e0389ebd24fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:34.860496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f03c6c-59de-4b5e-b5fa-e0389ebd24fe admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.870438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1302/2581] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:41:34.874090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f03c6c-59de-4b5e-b5fa-e0389ebd24fe admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:34.884025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f03c6c-59de-4b5e-b5fa-e0389ebd24fe admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:34.895830 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b4789a7544c43a29e41149af2ad2c68', 'request_id': 'req-13c99cb2-c60e-460e-abd4-969efc4fbdd1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.938647 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66602fd5-d3aa-49d6-8bc4-9181b68d3eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:34.939314 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66602fd5-d3aa-49d6-8bc4-9181b68d3eab admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.939596 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66602fd5-d3aa-49d6-8bc4-9181b68d3eab admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.939872 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66602fd5-d3aa-49d6-8bc4-9181b68d3eab admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.940623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66602fd5-d3aa-49d6-8bc4-9181b68d3eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:34.942216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66602fd5-d3aa-49d6-8bc4-9181b68d3eab admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.952962 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1303/2582] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:41:34.958205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66602fd5-d3aa-49d6-8bc4-9181b68d3eab admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:34.968731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66602fd5-d3aa-49d6-8bc4-9181b68d3eab admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:34.973334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea527a5-c8c2-45c3-bfba-cafaa7d4a502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:34.974216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea527a5-c8c2-45c3-bfba-cafaa7d4a502 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:34.975760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea527a5-c8c2-45c3-bfba-cafaa7d4a502 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:34.975760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea527a5-c8c2-45c3-bfba-cafaa7d4a502 admin admin] PATH_INFO: `/v3/projects/ddd3273a2f4343288d3b2ad3ce000ffb/users/af873b5c174b469185effca32a618a7b/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:34.990533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea527a5-c8c2-45c3-bfba-cafaa7d4a502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd3273a2f4343288d3b2ad3ce000ffb, user_id=af873b5c174b469185effca32a618a7b, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:34.990732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b4789a7544c43a29e41149af2ad2c68', 'request_id': 'req-13c99cb2-c60e-460e-abd4-969efc4fbdd1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:34.993198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea527a5-c8c2-45c3-bfba-cafaa7d4a502 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:34.998292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b4789a7544c43a29e41149af2ad2c68', 'request_id': 'req-13c99cb2-c60e-460e-abd4-969efc4fbdd1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:35.007170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:35.010856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] The token cache is being invalidate because project 4b4789a7544c43a29e41149af2ad2c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:35.011146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:35.011803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-13c99cb2-c60e-460e-abd4-969efc4fbdd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:35.013395 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1280/2583] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:34 2022] DELETE /identity/v3/projects/4b4789a7544c43a29e41149af2ad2c68 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:41:35.017977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524bdca-4fd2-4764-8e3f-213766207a09 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.023457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524bdca-4fd2-4764-8e3f-213766207a09 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.023457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524bdca-4fd2-4764-8e3f-213766207a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.035852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-5524bdca-4fd2-4764-8e3f-213766207a09 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:35.037083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5524bdca-4fd2-4764-8e3f-213766207a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:35.045044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1281/2584] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:35.054146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57a58-5fae-4f86-94c8-7c3eda97af3a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.054468 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57a58-5fae-4f86-94c8-7c3eda97af3a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.054747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57a58-5fae-4f86-94c8-7c3eda97af3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.073895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6cf57a58-5fae-4f86-94c8-7c3eda97af3a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:35.075114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6cf57a58-5fae-4f86-94c8-7c3eda97af3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:35.076687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1304/2585] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:34 2022] PUT /identity/v3/projects/ddd3273a2f4343288d3b2ad3ce000ffb/users/af873b5c174b469185effca32a618a7b/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:41:35.086889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6662-def6-45b8-a060-f766f3914288 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.087212 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6662-def6-45b8-a060-f766f3914288 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.087525 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6662-def6-45b8-a060-f766f3914288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.114199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-288f6662-def6-45b8-a060-f766f3914288 None None] MFA Rules not processed for user `af873b5c174b469185effca32a618a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:35.115411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-288f6662-def6-45b8-a060-f766f3914288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:35.127995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:35.143091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1282/2586] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:41:35.147327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf57a58-5fae-4f86-94c8-7c3eda97af3a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:35.156335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf57a58-5fae-4f86-94c8-7c3eda97af3a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:35.160955 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:35.161595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.162252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.162539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.163264 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:41:35.163571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:35.165190 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:35.177244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1283/2587] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:41:35.181187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:35.182583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1305/2588] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:41:35.191478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b2b20-7c71-4d29-9fd5-cfb7dacffae3 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:35.195003 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f6662-def6-45b8-a060-f766f3914288 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:35.196566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-288f6662-def6-45b8-a060-f766f3914288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:35.198597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8366d5b-b735-4448-a3bd-9425c94a4fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:35.199243 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366d5b-b735-4448-a3bd-9425c94a4fdf admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.199525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366d5b-b735-4448-a3bd-9425c94a4fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.199806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366d5b-b735-4448-a3bd-9425c94a4fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.200494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8366d5b-b735-4448-a3bd-9425c94a4fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:35.202448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8366d5b-b735-4448-a3bd-9425c94a4fdf admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:35.252790 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f6662-def6-45b8-a060-f766f3914288 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:35.256389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222d411-3644-4fce-87cf-c545f71b51b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c958b233374b4013a8bbe9efbed054a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:35.257085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222d411-3644-4fce-87cf-c545f71b51b4 service glance] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.257373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222d411-3644-4fce-87cf-c545f71b51b4 service glance] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.257658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222d411-3644-4fce-87cf-c545f71b51b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.272358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222d411-3644-4fce-87cf-c545f71b51b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:35.273948 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222d411-3644-4fce-87cf-c545f71b51b4 service glance] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:35.283467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1306/2589] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:41:35.335864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1284/2590] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:35 2022] POST /identity/v3/projects => generated 363 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:41:35.340883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222d411-3644-4fce-87cf-c545f71b51b4 service glance] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:35.353463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222d411-3644-4fce-87cf-c545f71b51b4 service glance] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:35.411405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513376b-962c-474f-8d1c-dfedc46fac1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:35.412080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513376b-962c-474f-8d1c-dfedc46fac1b admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.412366 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513376b-962c-474f-8d1c-dfedc46fac1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.412675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513376b-962c-474f-8d1c-dfedc46fac1b admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.413372 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513376b-962c-474f-8d1c-dfedc46fac1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:35.415274 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513376b-962c-474f-8d1c-dfedc46fac1b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:35.543549 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1307/2591] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:35 2022] POST /identity/v3/users => generated 282 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:41:35.544632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8366d5b-b735-4448-a3bd-9425c94a4fdf admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:35.554938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8366d5b-b735-4448-a3bd-9425c94a4fdf admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:35.559433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20391e8c-2319-40ab-b69a-5f2431a73db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:35.560084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20391e8c-2319-40ab-b69a-5f2431a73db8 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.560367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20391e8c-2319-40ab-b69a-5f2431a73db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.560678 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20391e8c-2319-40ab-b69a-5f2431a73db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.561394 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20391e8c-2319-40ab-b69a-5f2431a73db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:35.563557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20391e8c-2319-40ab-b69a-5f2431a73db8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:35.574431 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1285/2592] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:41:35.579258 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513376b-962c-474f-8d1c-dfedc46fac1b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:35.593493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513376b-962c-474f-8d1c-dfedc46fac1b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:35.599203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627f053-3161-4f62-b26a-379f66c3891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:35.600054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627f053-3161-4f62-b26a-379f66c3891f admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.600346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627f053-3161-4f62-b26a-379f66c3891f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.600666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627f053-3161-4f62-b26a-379f66c3891f admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.601431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627f053-3161-4f62-b26a-379f66c3891f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:35.603080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627f053-3161-4f62-b26a-379f66c3891f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:35.614408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1308/2593] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:35.618666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20391e8c-2319-40ab-b69a-5f2431a73db8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:35.629318 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20391e8c-2319-40ab-b69a-5f2431a73db8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:35.633947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eed344-308b-48b1-b3c4-5f7a0b94f46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:35.634819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eed344-308b-48b1-b3c4-5f7a0b94f46d admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.635105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eed344-308b-48b1-b3c4-5f7a0b94f46d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.635385 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eed344-308b-48b1-b3c4-5f7a0b94f46d admin admin] PATH_INFO: `/v3/projects/45c06c973dff4116b5a5786565e3cac3/users/88d4c684044b458bba3c06c6a86c639b/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.650158 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eed344-308b-48b1-b3c4-5f7a0b94f46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c06c973dff4116b5a5786565e3cac3, user_id=88d4c684044b458bba3c06c6a86c639b, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:35.652987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eed344-308b-48b1-b3c4-5f7a0b94f46d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:35.744747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1286/2594] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:35 2022] PUT /identity/v3/projects/45c06c973dff4116b5a5786565e3cac3/users/88d4c684044b458bba3c06c6a86c639b/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:35.755633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf58c33-53f0-46ef-9b28-f2e474507a9d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.756126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf58c33-53f0-46ef-9b28-f2e474507a9d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.756126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf58c33-53f0-46ef-9b28-f2e474507a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.783265 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-0bf58c33-53f0-46ef-9b28-f2e474507a9d None None] MFA Rules not processed for user `88d4c684044b458bba3c06c6a86c639b`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:35.784481 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0bf58c33-53f0-46ef-9b28-f2e474507a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:35.813761 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:35.853420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1309/2595] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:35.857846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eed344-308b-48b1-b3c4-5f7a0b94f46d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:35.920145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eed344-308b-48b1-b3c4-5f7a0b94f46d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:35.926254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5caa86-29ee-4ef4-8ede-d369d0e0c475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:35.926947 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5caa86-29ee-4ef4-8ede-d369d0e0c475 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:35.927380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5caa86-29ee-4ef4-8ede-d369d0e0c475 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:35.927624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5caa86-29ee-4ef4-8ede-d369d0e0c475 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:35.928409 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5caa86-29ee-4ef4-8ede-d369d0e0c475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:35.930294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5caa86-29ee-4ef4-8ede-d369d0e0c475 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:36.027524 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1287/2596] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:35 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:41:36.031144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf58c33-53f0-46ef-9b28-f2e474507a9d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:36.042328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf58c33-53f0-46ef-9b28-f2e474507a9d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:36.098246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2fa3e-df0b-4362-823c-8a0f2a8c1c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:36.098844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2fa3e-df0b-4362-823c-8a0f2a8c1c34 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:36.099131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2fa3e-df0b-4362-823c-8a0f2a8c1c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:36.099416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2fa3e-df0b-4362-823c-8a0f2a8c1c34 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:36.100104 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2fa3e-df0b-4362-823c-8a0f2a8c1c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:36.102098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2fa3e-df0b-4362-823c-8a0f2a8c1c34 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:36.197997 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1310/2597] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:36 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:36.201873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5caa86-29ee-4ef4-8ede-d369d0e0c475 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:36.215868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5caa86-29ee-4ef4-8ede-d369d0e0c475 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:36.221118 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722fe86-dd19-467a-a812-48228e3e08e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:36.221776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fe86-dd19-467a-a812-48228e3e08e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:36.222063 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fe86-dd19-467a-a812-48228e3e08e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:36.222346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fe86-dd19-467a-a812-48228e3e08e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:36.223075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722fe86-dd19-467a-a812-48228e3e08e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:36.224695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722fe86-dd19-467a-a812-48228e3e08e2 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:36.226837 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:36.237202 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1288/2598] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:36.241406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2fa3e-df0b-4362-823c-8a0f2a8c1c34 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:36.251353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2fa3e-df0b-4362-823c-8a0f2a8c1c34 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:36.256230 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a2501-c001-47cd-8b7e-3ec58689f756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:36.257139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2501-c001-47cd-8b7e-3ec58689f756 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:36.257445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2501-c001-47cd-8b7e-3ec58689f756 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:36.257705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2501-c001-47cd-8b7e-3ec58689f756 admin admin] PATH_INFO: `/v3/projects/a6e26f3d3cdd4bea834608af0357f43e/users/d9f3b2d981c646fa9a5d752ff355d63a/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:36.272940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a2501-c001-47cd-8b7e-3ec58689f756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e26f3d3cdd4bea834608af0357f43e, user_id=d9f3b2d981c646fa9a5d752ff355d63a, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:36.275636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a2501-c001-47cd-8b7e-3ec58689f756 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:36.386680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1311/2599] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:36 2022] PUT /identity/v3/projects/a6e26f3d3cdd4bea834608af0357f43e/users/d9f3b2d981c646fa9a5d752ff355d63a/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:36.390625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722fe86-dd19-467a-a812-48228e3e08e2 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:36.400963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722fe86-dd19-467a-a812-48228e3e08e2 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:36.457907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4613225-73c2-4478-b733-1c28d9630e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:36.458574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613225-73c2-4478-b733-1c28d9630e1f admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:36.458864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613225-73c2-4478-b733-1c28d9630e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:36.459141 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613225-73c2-4478-b733-1c28d9630e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:36.459863 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4613225-73c2-4478-b733-1c28d9630e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:36.461473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4613225-73c2-4478-b733-1c28d9630e1f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:36.471632 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1289/2600] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:36.475808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a2501-c001-47cd-8b7e-3ec58689f756 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:36.486418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a2501-c001-47cd-8b7e-3ec58689f756 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:36.491105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:36.491983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:36.492271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:36.493974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] PATH_INFO: `/v3/projects/a6e26f3d3cdd4bea834608af0357f43e/users/d9f3b2d981c646fa9a5d752ff355d63a/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:36.494518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e26f3d3cdd4bea834608af0357f43e, user_id=d9f3b2d981c646fa9a5d752ff355d63a, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:36.497187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:36.507387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1312/2601] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:36 2022] PUT /identity/v3/projects/a6e26f3d3cdd4bea834608af0357f43e/users/d9f3b2d981c646fa9a5d752ff355d63a/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:36.518015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871a72a-ecf6-4b17-bc2f-e635765a89e9 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:36.518443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871a72a-ecf6-4b17-bc2f-e635765a89e9 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:36.518778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871a72a-ecf6-4b17-bc2f-e635765a89e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:36.546311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8871a72a-ecf6-4b17-bc2f-e635765a89e9 None None] MFA Rules not processed for user `d9f3b2d981c646fa9a5d752ff355d63a`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:36.547525 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8871a72a-ecf6-4b17-bc2f-e635765a89e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:36.616757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1290/2602] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:36.628479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:36.630204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:36.691164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a1f61-133f-4e59-9563-6672883a2b77 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:36.695291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3aca8a-8296-41bd-b713-412914ff3aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:36.696064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3aca8a-8296-41bd-b713-412914ff3aae service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:36.696353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3aca8a-8296-41bd-b713-412914ff3aae service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:36.696672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3aca8a-8296-41bd-b713-412914ff3aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:36.706213 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3aca8a-8296-41bd-b713-412914ff3aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:36.707664 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3aca8a-8296-41bd-b713-412914ff3aae service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:36.717033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1313/2603] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:36.927378 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871a72a-ecf6-4b17-bc2f-e635765a89e9 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:36.937570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871a72a-ecf6-4b17-bc2f-e635765a89e9 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:36.942350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f2f47-4b5a-4ff6-965a-4eb2d5e05bad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:36.943155 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f2f47-4b5a-4ff6-965a-4eb2d5e05bad service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:36.943457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f2f47-4b5a-4ff6-965a-4eb2d5e05bad service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:36.943741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f2f47-4b5a-4ff6-965a-4eb2d5e05bad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:37.003137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f2f47-4b5a-4ff6-965a-4eb2d5e05bad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:37.004603 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f2f47-4b5a-4ff6-965a-4eb2d5e05bad service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:37.013448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1291/2604] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:36 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:46.712873 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:47.009400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:48.778397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3aca8a-8296-41bd-b713-412914ff3aae service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:48.840583 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3aca8a-8296-41bd-b713-412914ff3aae service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:48.845046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:48.845943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:48.846237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:48.846502 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] PATH_INFO: `/v3/users/88d4c684044b458bba3c06c6a86c639b` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:48.847668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d4c684044b458bba3c06c6a86c639b)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:48.849723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:49.043877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d4c684044b458bba3c06c6a86c639b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:49.056403 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d4c684044b458bba3c06c6a86c639b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:49.063246 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d4c684044b458bba3c06c6a86c639b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:49.063913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d4c684044b458bba3c06c6a86c639b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:49.064533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d4c684044b458bba3c06c6a86c639b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:49.193128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1314/2605] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:48 2022] DELETE /identity/v3/users/88d4c684044b458bba3c06c6a86c639b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:49.203959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9f90b-34f6-4bec-90d2-853aa35c333e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:49.204440 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9f90b-34f6-4bec-90d2-853aa35c333e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:49.204577 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9f90b-34f6-4bec-90d2-853aa35c333e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:49.221255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-bdc9f90b-34f6-4bec-90d2-853aa35c333e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:49.222461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-bdc9f90b-34f6-4bec-90d2-853aa35c333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:49.280404 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1292/2606] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:49.290203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:49.291699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:49.351128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c926341-8144-4c07-8df1-5d1cf28d992b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:49.354824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd671e01-5b33-4f56-98ba-d53e9637e694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:49.355477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd671e01-5b33-4f56-98ba-d53e9637e694 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:49.355766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd671e01-5b33-4f56-98ba-d53e9637e694 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:49.356052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd671e01-5b33-4f56-98ba-d53e9637e694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:49.365643 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd671e01-5b33-4f56-98ba-d53e9637e694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:49.367079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd671e01-5b33-4f56-98ba-d53e9637e694 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:49.376490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1315/2607] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:49.827671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9f90b-34f6-4bec-90d2-853aa35c333e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:49.829181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-bdc9f90b-34f6-4bec-90d2-853aa35c333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:49.842133 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9f90b-34f6-4bec-90d2-853aa35c333e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:49.847543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:49.848486 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:49.849507 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:49.849796 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] PATH_INFO: `/v3/projects/45c06c973dff4116b5a5786565e3cac3` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:49.850997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c06c973dff4116b5a5786565e3cac3)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:49.852959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:49.970799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c06c973dff4116b5a5786565e3cac3', 'request_id': 'req-91c6400c-0d4f-4caa-a9b9-4247acf60db1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.046528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c06c973dff4116b5a5786565e3cac3', 'request_id': 'req-91c6400c-0d4f-4caa-a9b9-4247acf60db1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.052386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c06c973dff4116b5a5786565e3cac3', 'request_id': 'req-91c6400c-0d4f-4caa-a9b9-4247acf60db1'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.063484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] The token cache is being invalidate because project 45c06c973dff4116b5a5786565e3cac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:50.064469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.064469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.065906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1293/2608] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:49 2022] DELETE /identity/v3/projects/45c06c973dff4116b5a5786565e3cac3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:50.070230 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd671e01-5b33-4f56-98ba-d53e9637e694 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:50.079755 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd671e01-5b33-4f56-98ba-d53e9637e694 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:50.098006 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:50.137180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:50.138109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:50.138393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:50.138671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] PATH_INFO: `/v3/users/d9f3b2d981c646fa9a5d752ff355d63a` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:50.139826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f3b2d981c646fa9a5d752ff355d63a)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:50.141783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:50.305218 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f3b2d981c646fa9a5d752ff355d63a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.317215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f3b2d981c646fa9a5d752ff355d63a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.324754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f3b2d981c646fa9a5d752ff355d63a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.325437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f3b2d981c646fa9a5d752ff355d63a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.326034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6f6b24a3-7f17-4324-8889-bb1dfd1c74a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f3b2d981c646fa9a5d752ff355d63a'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:50.420587 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1316/2609] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:50 2022] DELETE /identity/v3/users/d9f3b2d981c646fa9a5d752ff355d63a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:50.798136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:50.798136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:50.821135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6400c-0d4f-4caa-a9b9-4247acf60db1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:50.886145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:50.887054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:50.887433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:50.887715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] PATH_INFO: `/v3/projects/a6e26f3d3cdd4bea834608af0357f43e` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:50.889144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e26f3d3cdd4bea834608af0357f43e)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:50.891116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:50.993392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:51.009222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e26f3d3cdd4bea834608af0357f43e', 'request_id': 'req-1eeb00f8-722f-41fb-b520-e8fbc5122201'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:51.080566 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e26f3d3cdd4bea834608af0357f43e', 'request_id': 'req-1eeb00f8-722f-41fb-b520-e8fbc5122201'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:51.086315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e26f3d3cdd4bea834608af0357f43e', 'request_id': 'req-1eeb00f8-722f-41fb-b520-e8fbc5122201'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:51.097412 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] The token cache is being invalidate because project a6e26f3d3cdd4bea834608af0357f43e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:51.097703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:51.098325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1eeb00f8-722f-41fb-b520-e8fbc5122201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:51.100400 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1294/2610] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:50 2022] DELETE /identity/v3/projects/a6e26f3d3cdd4bea834608af0357f43e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:51.106559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0adc2d-a280-4993-9925-953b6de6c3da None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.107034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0adc2d-a280-4993-9925-953b6de6c3da None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.107144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0adc2d-a280-4993-9925-953b6de6c3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.125599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-cf0adc2d-a280-4993-9925-953b6de6c3da None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:51.126801 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-cf0adc2d-a280-4993-9925-953b6de6c3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:51.184490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1317/2611] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:51.193567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e485ff-510f-4fad-a59f-f8a5a2db87d5 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.193857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e485ff-510f-4fad-a59f-f8a5a2db87d5 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.194133 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e485ff-510f-4fad-a59f-f8a5a2db87d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.213634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-79e485ff-510f-4fad-a59f-f8a5a2db87d5 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:51.214759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-79e485ff-510f-4fad-a59f-f8a5a2db87d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:51.222500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1295/2612] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:51.227299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0adc2d-a280-4993-9925-953b6de6c3da None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:51.236914 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0adc2d-a280-4993-9925-953b6de6c3da None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:51.242171 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:51.242851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.243138 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.243424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.244147 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:41:51.244459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:51.246100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:51.257532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1318/2613] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:41:51.261543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e485ff-510f-4fad-a59f-f8a5a2db87d5 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:51.270482 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e485ff-510f-4fad-a59f-f8a5a2db87d5 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:51.275620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b432ef-0909-4237-a99b-80d5b2f9be93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:51.276253 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b432ef-0909-4237-a99b-80d5b2f9be93 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.276559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b432ef-0909-4237-a99b-80d5b2f9be93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.276839 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b432ef-0909-4237-a99b-80d5b2f9be93 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.277559 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b432ef-0909-4237-a99b-80d5b2f9be93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:51.279438 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b432ef-0909-4237-a99b-80d5b2f9be93 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:51.353237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1296/2614] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:51 2022] POST /identity/v3/projects => generated 371 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:51.357102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:51.367137 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5e54f-b2e2-497b-b44e-2346451ba821 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:51.393011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:51.424731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa6f37-d695-4677-95a6-982f93fae37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:51.425512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa6f37-d695-4677-95a6-982f93fae37f admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.425797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa6f37-d695-4677-95a6-982f93fae37f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.426072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa6f37-d695-4677-95a6-982f93fae37f admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.426749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa6f37-d695-4677-95a6-982f93fae37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:51.428678 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa6f37-d695-4677-95a6-982f93fae37f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:51.526950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1319/2615] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:51 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:51.531027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b432ef-0909-4237-a99b-80d5b2f9be93 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:51.541765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b432ef-0909-4237-a99b-80d5b2f9be93 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:51.546308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dd7b6-2f35-48ce-ae62-37e03aa410e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:51.546954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd7b6-2f35-48ce-ae62-37e03aa410e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.547238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd7b6-2f35-48ce-ae62-37e03aa410e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.547516 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd7b6-2f35-48ce-ae62-37e03aa410e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.548244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dd7b6-2f35-48ce-ae62-37e03aa410e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:51.549880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dd7b6-2f35-48ce-ae62-37e03aa410e8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:51.559969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1297/2616] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:51.563897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa6f37-d695-4677-95a6-982f93fae37f admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:51.574168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa6f37-d695-4677-95a6-982f93fae37f admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:51.578781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548927e8-20e0-4f05-9923-6d1baa66cd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:51.579437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548927e8-20e0-4f05-9923-6d1baa66cd47 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.579727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548927e8-20e0-4f05-9923-6d1baa66cd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.580011 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548927e8-20e0-4f05-9923-6d1baa66cd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.580787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548927e8-20e0-4f05-9923-6d1baa66cd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:51.582402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548927e8-20e0-4f05-9923-6d1baa66cd47 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:51.593325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1320/2617] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:51.597371 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dd7b6-2f35-48ce-ae62-37e03aa410e8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:51.607985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dd7b6-2f35-48ce-ae62-37e03aa410e8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:51.612639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:51.613514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.613802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.614088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] PATH_INFO: `/v3/projects/dbc28723fe794cf3aedaa29884125459/users/b237b17beb3a48578fbc7f2950c2c633/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.628317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc28723fe794cf3aedaa29884125459, user_id=b237b17beb3a48578fbc7f2950c2c633, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:51.631523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:51.723448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1298/2618] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:51 2022] PUT /identity/v3/projects/dbc28723fe794cf3aedaa29884125459/users/b237b17beb3a48578fbc7f2950c2c633/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:51.733656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd5b2c-9f9c-480d-844c-b375c46bf210 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.733952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd5b2c-9f9c-480d-844c-b375c46bf210 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.734237 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd5b2c-9f9c-480d-844c-b375c46bf210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.760729 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7cfd5b2c-9f9c-480d-844c-b375c46bf210 None None] MFA Rules not processed for user `b237b17beb3a48578fbc7f2950c2c633`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:51.761934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7cfd5b2c-9f9c-480d-844c-b375c46bf210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:51.828883 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1321/2619] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:51.840369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:51.842109 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:51.898410 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ef2a9-918a-44ce-a4d6-d7565416ae0b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:51.902375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917ba7e-769b-41da-a91c-25b5308cb970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:51.903025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917ba7e-769b-41da-a91c-25b5308cb970 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:51.903314 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917ba7e-769b-41da-a91c-25b5308cb970 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:51.903591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917ba7e-769b-41da-a91c-25b5308cb970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:51.912177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917ba7e-769b-41da-a91c-25b5308cb970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:51.913623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917ba7e-769b-41da-a91c-25b5308cb970 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:51.922681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1299/2620] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:51.941492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd5b2c-9f9c-480d-844c-b375c46bf210 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:52.005027 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd5b2c-9f9c-480d-844c-b375c46bf210 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:52.010304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:52.011179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:52.011465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:52.011746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] PATH_INFO: `/v3/users/b237b17beb3a48578fbc7f2950c2c633` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:52.012981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b237b17beb3a48578fbc7f2950c2c633)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:52.014882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:52.181285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:52.187016 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b237b17beb3a48578fbc7f2950c2c633'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:52.199387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b237b17beb3a48578fbc7f2950c2c633'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:52.205995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b237b17beb3a48578fbc7f2950c2c633'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:52.208224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b237b17beb3a48578fbc7f2950c2c633'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:52.208224 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b237b17beb3a48578fbc7f2950c2c633'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:52.274803 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1322/2621] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:41:51 2022] DELETE /identity/v3/users/b237b17beb3a48578fbc7f2950c2c633 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:52.279375 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e5e14-6b17-46af-9c4a-0c0360a7de0a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:52.279651 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e5e14-6b17-46af-9c4a-0c0360a7de0a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:52.286273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e5e14-6b17-46af-9c4a-0c0360a7de0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:52.296538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c10e5e14-6b17-46af-9c4a-0c0360a7de0a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:52.297778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c10e5e14-6b17-46af-9c4a-0c0360a7de0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:52.344071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:52.356235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1300/2622] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:52.366490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:52.367892 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:52.425536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcddccb-57e6-4c61-b776-77f4f0acc54b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:52.429739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d07cb-4f42-439e-a66a-71ea83f8ac68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:52.430430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d07cb-4f42-439e-a66a-71ea83f8ac68 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:52.430714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d07cb-4f42-439e-a66a-71ea83f8ac68 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:52.430995 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d07cb-4f42-439e-a66a-71ea83f8ac68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:52.440391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d07cb-4f42-439e-a66a-71ea83f8ac68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:52.441949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d07cb-4f42-439e-a66a-71ea83f8ac68 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:52.451831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1323/2623] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:41:52.865106 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e5e14-6b17-46af-9c4a-0c0360a7de0a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:52.866781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c10e5e14-6b17-46af-9c4a-0c0360a7de0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:52.883743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e5e14-6b17-46af-9c4a-0c0360a7de0a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:52.884676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:52.885550 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:52.885834 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:52.886113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] PATH_INFO: `/v3/projects/dbc28723fe794cf3aedaa29884125459` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:52.887312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc28723fe794cf3aedaa29884125459)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:52.889287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:53.024705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc28723fe794cf3aedaa29884125459', 'request_id': 'req-d3d72f7f-896e-418c-9537-8e43da9635d6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:53.100446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc28723fe794cf3aedaa29884125459', 'request_id': 'req-d3d72f7f-896e-418c-9537-8e43da9635d6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:53.106186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc28723fe794cf3aedaa29884125459', 'request_id': 'req-d3d72f7f-896e-418c-9537-8e43da9635d6'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:53.117489 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] The token cache is being invalidate because project dbc28723fe794cf3aedaa29884125459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:41:53.117780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:53.118505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d3d72f7f-896e-418c-9537-8e43da9635d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:41:53.120139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1301/2624] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:41:52 2022] DELETE /identity/v3/projects/dbc28723fe794cf3aedaa29884125459 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:53.125103 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae3fb-46b1-4592-a6d5-0d051d168090 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.125398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae3fb-46b1-4592-a6d5-0d051d168090 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.125709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae3fb-46b1-4592-a6d5-0d051d168090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.144362 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-514ae3fb-46b1-4592-a6d5-0d051d168090 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:53.146253 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-514ae3fb-46b1-4592-a6d5-0d051d168090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:53.210312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1324/2625] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:53.218061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005e87-2d2c-4137-a72e-caa52386511a None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.218359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005e87-2d2c-4137-a72e-caa52386511a None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.218641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005e87-2d2c-4137-a72e-caa52386511a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.237120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c9005e87-2d2c-4137-a72e-caa52386511a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:53.238334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c9005e87-2d2c-4137-a72e-caa52386511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:53.246870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1302/2626] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:53.251387 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ae3fb-46b1-4592-a6d5-0d051d168090 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:53.262043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ae3fb-46b1-4592-a6d5-0d051d168090 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:53.267010 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:53.267706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.267993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.268275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.269083 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:41:53.269393 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:53.271612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:53.288055 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1325/2627] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:41:53.289124 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9005e87-2d2c-4137-a72e-caa52386511a None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:53.299099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9005e87-2d2c-4137-a72e-caa52386511a None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:53.303845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0b0a-02f2-4bfb-a031-448e39154c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:53.304759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0b0a-02f2-4bfb-a031-448e39154c64 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.305052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0b0a-02f2-4bfb-a031-448e39154c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.305334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0b0a-02f2-4bfb-a031-448e39154c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.306051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d0b0a-02f2-4bfb-a031-448e39154c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:53.308011 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d0b0a-02f2-4bfb-a031-448e39154c64 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:53.426195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1303/2628] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:41:53 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:41:53.427658 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:53.438434 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579086f-28b1-429c-854b-72de9507352b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:53.485141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:41:53.498404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0c0f-bc02-473e-89e2-80cda5bef97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:53.499571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0c0f-bc02-473e-89e2-80cda5bef97d admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.499857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0c0f-bc02-473e-89e2-80cda5bef97d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.500164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0c0f-bc02-473e-89e2-80cda5bef97d admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.500890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf0c0f-bc02-473e-89e2-80cda5bef97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:53.502901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf0c0f-bc02-473e-89e2-80cda5bef97d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:53.621278 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1326/2629] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:41:53 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:41:53.625312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0b0a-02f2-4bfb-a031-448e39154c64 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:53.636152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0b0a-02f2-4bfb-a031-448e39154c64 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:53.640672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40577c-8a91-40f7-a66a-deb6eb9e427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:53.641326 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40577c-8a91-40f7-a66a-deb6eb9e427a admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.641652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40577c-8a91-40f7-a66a-deb6eb9e427a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.641933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40577c-8a91-40f7-a66a-deb6eb9e427a admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.642660 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40577c-8a91-40f7-a66a-deb6eb9e427a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:53.644241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40577c-8a91-40f7-a66a-deb6eb9e427a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:53.654621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1304/2630] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:53.658653 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0c0f-bc02-473e-89e2-80cda5bef97d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:53.668850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0c0f-bc02-473e-89e2-80cda5bef97d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:53.674175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3885db-7720-4598-86d9-9bd2dc444dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:53.674839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3885db-7720-4598-86d9-9bd2dc444dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.675134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3885db-7720-4598-86d9-9bd2dc444dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.675413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3885db-7720-4598-86d9-9bd2dc444dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.676150 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3885db-7720-4598-86d9-9bd2dc444dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:53.678295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3885db-7720-4598-86d9-9bd2dc444dd0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:53.689168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1327/2631] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:41:53.693868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40577c-8a91-40f7-a66a-deb6eb9e427a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:53.704235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40577c-8a91-40f7-a66a-deb6eb9e427a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:53.708910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:53.709777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.710091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.710414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] PATH_INFO: `/v3/projects/c12961c0fc9744afa4f0d90b06c1cf3d/users/3ccb11538d7546bfbb384be32f7c4b98/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.725126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12961c0fc9744afa4f0d90b06c1cf3d, user_id=3ccb11538d7546bfbb384be32f7c4b98, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:53.727822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:53.798055 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1305/2632] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:41:53 2022] PUT /identity/v3/projects/c12961c0fc9744afa4f0d90b06c1cf3d/users/3ccb11538d7546bfbb384be32f7c4b98/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:41:53.810287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445be77-42db-4d03-8faf-3354f3026d96 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.810581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445be77-42db-4d03-8faf-3354f3026d96 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.810864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445be77-42db-4d03-8faf-3354f3026d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.837495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-6445be77-42db-4d03-8faf-3354f3026d96 None None] MFA Rules not processed for user `3ccb11538d7546bfbb384be32f7c4b98`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:41:53.838694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6445be77-42db-4d03-8faf-3354f3026d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:53.903646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1328/2633] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:41:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:41:53.915040 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:41:53.916797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:41:53.971979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77130744-d32b-4aa2-b6f8-83ef025da43e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:41:53.976361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbf7b4-1e40-4b0b-bcd6-0f4a99cd22b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c958b233374b4013a8bbe9efbed054a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:41:53.977030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbf7b4-1e40-4b0b-bcd6-0f4a99cd22b9 service glance] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:41:53.977312 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbf7b4-1e40-4b0b-bcd6-0f4a99cd22b9 service glance] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:41:53.977583 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbf7b4-1e40-4b0b-bcd6-0f4a99cd22b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:41:53.986015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbf7b4-1e40-4b0b-bcd6-0f4a99cd22b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:41:53.987419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbf7b4-1e40-4b0b-bcd6-0f4a99cd22b9 service glance] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:41:53.996996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1306/2634] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:41:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:03.900816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:03.992794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:06.652632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445be77-42db-4d03-8faf-3354f3026d96 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:06.654033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6445be77-42db-4d03-8faf-3354f3026d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:06.719947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445be77-42db-4d03-8faf-3354f3026d96 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:06.724871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:06.726029 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:06.726713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:06.726998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] PATH_INFO: `/v3/users/af873b5c174b469185effca32a618a7b` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:06.728199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af873b5c174b469185effca32a618a7b)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:06.730184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:06.945787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af873b5c174b469185effca32a618a7b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:06.958287 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af873b5c174b469185effca32a618a7b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:06.965422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af873b5c174b469185effca32a618a7b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:06.966094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af873b5c174b469185effca32a618a7b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:06.966699 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af873b5c174b469185effca32a618a7b'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:07.040250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1329/2635] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:06 2022] DELETE /identity/v3/users/af873b5c174b469185effca32a618a7b => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:07.044763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e2f3c-2754-4d8b-8f8b-855b3c074738 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:07.045222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e2f3c-2754-4d8b-8f8b-855b3c074738 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:07.045319 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e2f3c-2754-4d8b-8f8b-855b3c074738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:07.063209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-2d2e2f3c-2754-4d8b-8f8b-855b3c074738 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:07.064515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2d2e2f3c-2754-4d8b-8f8b-855b3c074738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:07.123160 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1307/2636] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:07.133977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:07.136105 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:07.192248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcad57-3c96-42b4-b152-113728d78e75 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:07.196334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af5da-28f3-4703-9eab-d0705a4385d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:07.196946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af5da-28f3-4703-9eab-d0705a4385d7 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:07.197228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af5da-28f3-4703-9eab-d0705a4385d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:07.198061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af5da-28f3-4703-9eab-d0705a4385d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:07.207009 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af5da-28f3-4703-9eab-d0705a4385d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:07.208424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af5da-28f3-4703-9eab-d0705a4385d7 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:07.218367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1330/2637] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:07.589366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e2f3c-2754-4d8b-8f8b-855b3c074738 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:07.589366 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2d2e2f3c-2754-4d8b-8f8b-855b3c074738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:07.604799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e2f3c-2754-4d8b-8f8b-855b3c074738 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:07.609959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:07.610889 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:07.611347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:07.611630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] PATH_INFO: `/v3/projects/ddd3273a2f4343288d3b2ad3ce000ffb` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:07.613110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddd3273a2f4343288d3b2ad3ce000ffb)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:07.615236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:07.749293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddd3273a2f4343288d3b2ad3ce000ffb', 'request_id': 'req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:07.814713 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddd3273a2f4343288d3b2ad3ce000ffb', 'request_id': 'req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:07.821681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddd3273a2f4343288d3b2ad3ce000ffb', 'request_id': 'req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:07.837654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] The token cache is being invalidate because project ddd3273a2f4343288d3b2ad3ce000ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:07.838527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:07.839112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-cc80a9a0-c2b9-4973-9ae6-918541c6d63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:07.840753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1308/2638] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:07 2022] DELETE /identity/v3/projects/ddd3273a2f4343288d3b2ad3ce000ffb => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:07.845575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6577e8-63d8-4f06-85a1-7382c8967b19 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:07.846046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6577e8-63d8-4f06-85a1-7382c8967b19 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:07.846160 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6577e8-63d8-4f06-85a1-7382c8967b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:07.864662 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-ad6577e8-63d8-4f06-85a1-7382c8967b19 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:07.865866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ad6577e8-63d8-4f06-85a1-7382c8967b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:07.883109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:07.925249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1331/2639] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:07.934113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cf1e7-4757-4341-a33c-6730c26d21fc None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:07.934413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cf1e7-4757-4341-a33c-6730c26d21fc None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:07.934705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cf1e7-4757-4341-a33c-6730c26d21fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:07.953256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-3e2cf1e7-4757-4341-a33c-6730c26d21fc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:07.954532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3e2cf1e7-4757-4341-a33c-6730c26d21fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:07.962474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1309/2640] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:07.967215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6577e8-63d8-4f06-85a1-7382c8967b19 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:07.976885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6577e8-63d8-4f06-85a1-7382c8967b19 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:07.982795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:07.983480 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:07.983768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:07.984052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:07.984808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:42:07.985117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:07.986781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:07.998251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1332/2641] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:42:08.002369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cf1e7-4757-4341-a33c-6730c26d21fc None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:08.012113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cf1e7-4757-4341-a33c-6730c26d21fc None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:08.016850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f597-0b62-441b-b6c9-fe89c181877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:08.017492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f597-0b62-441b-b6c9-fe89c181877a admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.017783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f597-0b62-441b-b6c9-fe89c181877a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.018091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f597-0b62-441b-b6c9-fe89c181877a admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.018777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5f597-0b62-441b-b6c9-fe89c181877a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:08.020699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5f597-0b62-441b-b6c9-fe89c181877a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:08.141516 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1310/2642] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:42:08 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:08.145694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:08.156553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d64f6-01c4-4a6e-97da-f16c5466712b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:08.212411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e9596-a19a-4841-a6b8-cc41f4a369d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:08.213739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e9596-a19a-4841-a6b8-cc41f4a369d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.214051 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e9596-a19a-4841-a6b8-cc41f4a369d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.214320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e9596-a19a-4841-a6b8-cc41f4a369d6 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.215018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e9596-a19a-4841-a6b8-cc41f4a369d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:08.216978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e9596-a19a-4841-a6b8-cc41f4a369d6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:08.297797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1333/2643] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:42:08 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:08.301873 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f597-0b62-441b-b6c9-fe89c181877a admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:08.312075 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f597-0b62-441b-b6c9-fe89c181877a admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:08.316763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611533ba-8104-4053-8d2d-618dd0d55dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:08.317442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611533ba-8104-4053-8d2d-618dd0d55dfb admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.317726 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611533ba-8104-4053-8d2d-618dd0d55dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.318005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611533ba-8104-4053-8d2d-618dd0d55dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.318731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611533ba-8104-4053-8d2d-618dd0d55dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:08.320712 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611533ba-8104-4053-8d2d-618dd0d55dfb admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:08.331279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1311/2644] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:08.335185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e9596-a19a-4841-a6b8-cc41f4a369d6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:08.345860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e9596-a19a-4841-a6b8-cc41f4a369d6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:08.351261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055096d-881d-4776-aa14-3139e340827e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:08.351953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055096d-881d-4776-aa14-3139e340827e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.352244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055096d-881d-4776-aa14-3139e340827e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.352620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055096d-881d-4776-aa14-3139e340827e admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.353288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055096d-881d-4776-aa14-3139e340827e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:08.354938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055096d-881d-4776-aa14-3139e340827e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:08.364945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1334/2645] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:08.368845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611533ba-8104-4053-8d2d-618dd0d55dfb admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:08.379148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611533ba-8104-4053-8d2d-618dd0d55dfb admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:08.383655 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:08.384516 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.385435 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.385737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] PATH_INFO: `/v3/projects/9b8422c1ec9349ef8054b67f89ba8357/users/2829c86ec5474ee1944774d55528850f/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.400007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8422c1ec9349ef8054b67f89ba8357, user_id=2829c86ec5474ee1944774d55528850f, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:08.402688 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:08.494935 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1312/2646] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:42:08 2022] PUT /identity/v3/projects/9b8422c1ec9349ef8054b67f89ba8357/users/2829c86ec5474ee1944774d55528850f/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:42:08.554928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915bba9-1df8-4896-b296-e30b15760677 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.555123 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915bba9-1df8-4896-b296-e30b15760677 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.555400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915bba9-1df8-4896-b296-e30b15760677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.583856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d915bba9-1df8-4896-b296-e30b15760677 None None] MFA Rules not processed for user `2829c86ec5474ee1944774d55528850f`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:08.585080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d915bba9-1df8-4896-b296-e30b15760677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:08.650410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1335/2647] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:08.661672 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:08.663342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:08.684997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:08.719246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89ba94-b4d8-4a79-97a4-3467ad98d7f8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:08.722832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38506ca6-3aea-4433-b9b0-2a195fd66471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:08.723470 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38506ca6-3aea-4433-b9b0-2a195fd66471 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.723754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38506ca6-3aea-4433-b9b0-2a195fd66471 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.724041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38506ca6-3aea-4433-b9b0-2a195fd66471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.732984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38506ca6-3aea-4433-b9b0-2a195fd66471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:08.734397 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38506ca6-3aea-4433-b9b0-2a195fd66471 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:08.743057 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1313/2648] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:08.767592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915bba9-1df8-4896-b296-e30b15760677 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:08.786155 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:08.829130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915bba9-1df8-4896-b296-e30b15760677 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:08.833607 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:08.834496 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.834786 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.835068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] PATH_INFO: `/v3/users/2829c86ec5474ee1944774d55528850f` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.836261 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2829c86ec5474ee1944774d55528850f)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:08.838203 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:09.002370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2829c86ec5474ee1944774d55528850f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.014190 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2829c86ec5474ee1944774d55528850f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.020934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2829c86ec5474ee1944774d55528850f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.021612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2829c86ec5474ee1944774d55528850f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.023163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2829c86ec5474ee1944774d55528850f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.094428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1336/2649] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:08 2022] DELETE /identity/v3/users/2829c86ec5474ee1944774d55528850f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:09.098864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f19b-a0ac-4f93-8f87-df27ea933d49 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:09.099159 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f19b-a0ac-4f93-8f87-df27ea933d49 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:09.099447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f19b-a0ac-4f93-8f87-df27ea933d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:09.116855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9e72f19b-a0ac-4f93-8f87-df27ea933d49 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:09.118050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9e72f19b-a0ac-4f93-8f87-df27ea933d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:09.176666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1314/2650] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:09.186319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:09.187706 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:09.244347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510a89-ff7a-4b61-beef-e49d027e572d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:09.248000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9ce6a-646f-4188-95e2-f0db20b06be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:09.248667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9ce6a-646f-4188-95e2-f0db20b06be4 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:09.248954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9ce6a-646f-4188-95e2-f0db20b06be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:09.249229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9ce6a-646f-4188-95e2-f0db20b06be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:09.258056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9ce6a-646f-4188-95e2-f0db20b06be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:09.259476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9ce6a-646f-4188-95e2-f0db20b06be4 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:09.268839 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1337/2651] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:09.654606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72f19b-a0ac-4f93-8f87-df27ea933d49 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:09.656349 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9e72f19b-a0ac-4f93-8f87-df27ea933d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:09.668751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72f19b-a0ac-4f93-8f87-df27ea933d49 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:09.674067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:09.674946 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:09.675237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:09.675528 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] PATH_INFO: `/v3/projects/9b8422c1ec9349ef8054b67f89ba8357` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:09.676795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8422c1ec9349ef8054b67f89ba8357)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:09.678790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:09.787053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8422c1ec9349ef8054b67f89ba8357', 'request_id': 'req-bb4580e1-1afd-4a37-9578-b4c04aa946e3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.852310 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8422c1ec9349ef8054b67f89ba8357', 'request_id': 'req-bb4580e1-1afd-4a37-9578-b4c04aa946e3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.858505 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8422c1ec9349ef8054b67f89ba8357', 'request_id': 'req-bb4580e1-1afd-4a37-9578-b4c04aa946e3'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.870236 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] The token cache is being invalidate because project 9b8422c1ec9349ef8054b67f89ba8357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:09.870529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.871194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bb4580e1-1afd-4a37-9578-b4c04aa946e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:09.872781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1315/2652] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:09 2022] DELETE /identity/v3/projects/9b8422c1ec9349ef8054b67f89ba8357 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:09.877785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29234633-f2f9-4af0-8dc9-2c43d2ae710e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:09.878193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29234633-f2f9-4af0-8dc9-2c43d2ae710e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:09.878467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29234633-f2f9-4af0-8dc9-2c43d2ae710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:09.896824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-29234633-f2f9-4af0-8dc9-2c43d2ae710e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:09.898091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-29234633-f2f9-4af0-8dc9-2c43d2ae710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:09.933397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:09.962450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1338/2653] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:09.969741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde2d-836b-42e7-a7e5-370b3b4a5c2e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:09.970030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde2d-836b-42e7-a7e5-370b3b4a5c2e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:09.970309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde2d-836b-42e7-a7e5-370b3b4a5c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:09.988091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d6cfde2d-836b-42e7-a7e5-370b3b4a5c2e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:09.989333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d6cfde2d-836b-42e7-a7e5-370b3b4a5c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:09.997727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1316/2654] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:10.002034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29234633-f2f9-4af0-8dc9-2c43d2ae710e None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:10.012117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29234633-f2f9-4af0-8dc9-2c43d2ae710e None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:10.017094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:10.017741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.018024 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.018303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.019047 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:42:10.019354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:10.020973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:10.032238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1339/2655] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:42:10.036305 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfde2d-836b-42e7-a7e5-370b3b4a5c2e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:10.045458 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfde2d-836b-42e7-a7e5-370b3b4a5c2e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:10.050045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc7d5a-87eb-4b2a-9984-6554b1fc2501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:10.050694 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc7d5a-87eb-4b2a-9984-6554b1fc2501 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.050982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc7d5a-87eb-4b2a-9984-6554b1fc2501 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.051266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc7d5a-87eb-4b2a-9984-6554b1fc2501 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.051965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc7d5a-87eb-4b2a-9984-6554b1fc2501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:10.053890 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc7d5a-87eb-4b2a-9984-6554b1fc2501 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:10.122630 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1317/2656] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:42:10 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:42:10.126532 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:10.136971 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ada1f7-a784-443a-8add-c3d82d66e402 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:10.191013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab0e58-2e34-4cab-a5a3-b04e02b20c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:10.191614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab0e58-2e34-4cab-a5a3-b04e02b20c04 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.191903 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab0e58-2e34-4cab-a5a3-b04e02b20c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.192299 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab0e58-2e34-4cab-a5a3-b04e02b20c04 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.193031 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab0e58-2e34-4cab-a5a3-b04e02b20c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:10.194943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab0e58-2e34-4cab-a5a3-b04e02b20c04 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:10.266180 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1340/2657] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:42:10 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:10.270243 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc7d5a-87eb-4b2a-9984-6554b1fc2501 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:10.280481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc7d5a-87eb-4b2a-9984-6554b1fc2501 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:10.285111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcd3e1-fa50-4b5e-a08a-9539fc7dc279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:10.285758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcd3e1-fa50-4b5e-a08a-9539fc7dc279 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.286036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcd3e1-fa50-4b5e-a08a-9539fc7dc279 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.286309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcd3e1-fa50-4b5e-a08a-9539fc7dc279 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.287024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dcd3e1-fa50-4b5e-a08a-9539fc7dc279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:10.288621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dcd3e1-fa50-4b5e-a08a-9539fc7dc279 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:10.299367 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1318/2658] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:10.303338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab0e58-2e34-4cab-a5a3-b04e02b20c04 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:10.313854 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab0e58-2e34-4cab-a5a3-b04e02b20c04 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:10.318411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2903f-7316-4bd0-b32d-43600467b7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:10.319063 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2903f-7316-4bd0-b32d-43600467b7ac admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.319351 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2903f-7316-4bd0-b32d-43600467b7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.319633 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2903f-7316-4bd0-b32d-43600467b7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.320645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2903f-7316-4bd0-b32d-43600467b7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:10.322239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2903f-7316-4bd0-b32d-43600467b7ac admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:10.331972 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1341/2659] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:10.335888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcd3e1-fa50-4b5e-a08a-9539fc7dc279 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:10.345585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcd3e1-fa50-4b5e-a08a-9539fc7dc279 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:10.350115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:10.350981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.351263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.351540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] PATH_INFO: `/v3/projects/3a122fcfab7f418baf78d7b742fa20db/users/517928d85b764c089c6dd5b2d088639c/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.353722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:10.366430 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a122fcfab7f418baf78d7b742fa20db, user_id=517928d85b764c089c6dd5b2d088639c, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:10.369113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:10.425538 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1319/2660] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:42:10 2022] PUT /identity/v3/projects/3a122fcfab7f418baf78d7b742fa20db/users/517928d85b764c089c6dd5b2d088639c/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:10.430922 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dd98-97b0-4a12-bc2b-0c86545f9fb5 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.431293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dd98-97b0-4a12-bc2b-0c86545f9fb5 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.431578 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dd98-97b0-4a12-bc2b-0c86545f9fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.450310 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d480dd98-97b0-4a12-bc2b-0c86545f9fb5 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:10.451440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d480dd98-97b0-4a12-bc2b-0c86545f9fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:10.511085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1342/2661] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:10.521364 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:10.530544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:10.582614 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d9ad0-77f4-4901-80c3-4d8fe5d0bd69 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:10.586496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e7294-86e6-4721-b780-e09c17ad14ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:10.587181 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e7294-86e6-4721-b780-e09c17ad14ea service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.587494 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e7294-86e6-4721-b780-e09c17ad14ea service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.587803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e7294-86e6-4721-b780-e09c17ad14ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.598142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e7294-86e6-4721-b780-e09c17ad14ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:10.599567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e7294-86e6-4721-b780-e09c17ad14ea service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:10.609144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1320/2662] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:13.451837 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480dd98-97b0-4a12-bc2b-0c86545f9fb5 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:13.454674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d480dd98-97b0-4a12-bc2b-0c86545f9fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:13.468730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480dd98-97b0-4a12-bc2b-0c86545f9fb5 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:13.473691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:13.474793 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:13.475186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:13.475570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] PATH_INFO: `/v3/users/3ccb11538d7546bfbb384be32f7c4b98` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:13.477516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ccb11538d7546bfbb384be32f7c4b98)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:13.478990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:13.678909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:13.685457 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ccb11538d7546bfbb384be32f7c4b98'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:13.700456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ccb11538d7546bfbb384be32f7c4b98'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:13.708028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ccb11538d7546bfbb384be32f7c4b98'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:13.708824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ccb11538d7546bfbb384be32f7c4b98'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:13.709472 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ccb11538d7546bfbb384be32f7c4b98'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:13.809760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1343/2663] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:13 2022] DELETE /identity/v3/users/3ccb11538d7546bfbb384be32f7c4b98 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:13.814184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ee3a-9161-47da-9c14-51bb6844ff8e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:13.815172 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ee3a-9161-47da-9c14-51bb6844ff8e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:13.815437 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ee3a-9161-47da-9c14-51bb6844ff8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:13.834071 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c0f7ee3a-9161-47da-9c14-51bb6844ff8e None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:13.835304 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c0f7ee3a-9161-47da-9c14-51bb6844ff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:13.921368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1321/2664] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:13.932781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:13.935293 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:14.044735 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1b230-c252-42d6-b2d5-cba46c1b0b6f admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:14.049375 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2fb51-cfc2-4cbe-b910-621cfa78b80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:14.050249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2fb51-cfc2-4cbe-b910-621cfa78b80b service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:14.050718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2fb51-cfc2-4cbe-b910-621cfa78b80b service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:14.051784 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2fb51-cfc2-4cbe-b910-621cfa78b80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:14.066300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2fb51-cfc2-4cbe-b910-621cfa78b80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:14.068014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2fb51-cfc2-4cbe-b910-621cfa78b80b service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:14.093639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1344/2665] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:14.582280 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7ee3a-9161-47da-9c14-51bb6844ff8e None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:14.585004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c0f7ee3a-9161-47da-9c14-51bb6844ff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:14.602682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7ee3a-9161-47da-9c14-51bb6844ff8e None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:14.613523 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:14.614418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:14.614710 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:14.615358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] PATH_INFO: `/v3/projects/c12961c0fc9744afa4f0d90b06c1cf3d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:14.616249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12961c0fc9744afa4f0d90b06c1cf3d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:14.618313 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:14.746695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12961c0fc9744afa4f0d90b06c1cf3d', 'request_id': 'req-f2171e6f-bd89-42e8-9ee3-82054f11c795'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:14.832081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12961c0fc9744afa4f0d90b06c1cf3d', 'request_id': 'req-f2171e6f-bd89-42e8-9ee3-82054f11c795'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:14.839129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12961c0fc9744afa4f0d90b06c1cf3d', 'request_id': 'req-f2171e6f-bd89-42e8-9ee3-82054f11c795'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:14.851610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] The token cache is being invalidate because project c12961c0fc9744afa4f0d90b06c1cf3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:14.852005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:14.852832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f2171e6f-bd89-42e8-9ee3-82054f11c795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:14.855146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1322/2666] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:14 2022] DELETE /identity/v3/projects/c12961c0fc9744afa4f0d90b06c1cf3d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:14.860710 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855b20-48dd-463c-8ca3-42e81e0f2b17 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:14.861249 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855b20-48dd-463c-8ca3-42e81e0f2b17 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:14.862111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855b20-48dd-463c-8ca3-42e81e0f2b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:14.883804 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f5855b20-48dd-463c-8ca3-42e81e0f2b17 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:14.885209 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f5855b20-48dd-463c-8ca3-42e81e0f2b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:14.961338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1345/2667] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:42:14.969912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68aa46-e036-47b4-9703-f0a20c79f339 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:14.970674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68aa46-e036-47b4-9703-f0a20c79f339 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:14.971193 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68aa46-e036-47b4-9703-f0a20c79f339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:14.992122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-bf68aa46-e036-47b4-9703-f0a20c79f339 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:14.993570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-bf68aa46-e036-47b4-9703-f0a20c79f339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:15.017398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1323/2668] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:15.022185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855b20-48dd-463c-8ca3-42e81e0f2b17 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:15.037132 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855b20-48dd-463c-8ca3-42e81e0f2b17 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:15.043744 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:15.044425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:15.044756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:15.045040 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:15.045783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:42:15.046102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:15.053506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:15.076401 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1346/2669] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:42:15.080846 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68aa46-e036-47b4-9703-f0a20c79f339 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:15.085343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:15.092798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68aa46-e036-47b4-9703-f0a20c79f339 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:15.126219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aabe6f-5f52-44c2-bd17-6f8af6a3d48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:15.126219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aabe6f-5f52-44c2-bd17-6f8af6a3d48c admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:15.126219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aabe6f-5f52-44c2-bd17-6f8af6a3d48c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:15.126219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aabe6f-5f52-44c2-bd17-6f8af6a3d48c admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:15.128104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aabe6f-5f52-44c2-bd17-6f8af6a3d48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:15.135492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aabe6f-5f52-44c2-bd17-6f8af6a3d48c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:15.254714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1324/2670] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:42:15 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:15.259321 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:15.272887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022d4f8-9f5d-4b6b-aa0a-5cf7f05de7f6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:15.327778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:15.345062 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ffef0-b9db-479f-a982-2b80e395a059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:15.345838 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ffef0-b9db-479f-a982-2b80e395a059 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:15.346197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ffef0-b9db-479f-a982-2b80e395a059 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:15.346542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ffef0-b9db-479f-a982-2b80e395a059 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:15.347316 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ffef0-b9db-479f-a982-2b80e395a059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:15.349333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ffef0-b9db-479f-a982-2b80e395a059 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:15.450955 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1347/2671] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:42:15 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:15.452240 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aabe6f-5f52-44c2-bd17-6f8af6a3d48c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:15.466919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aabe6f-5f52-44c2-bd17-6f8af6a3d48c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:15.485203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa2e5b-722b-470d-8561-9ff23a87d3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:15.485203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa2e5b-722b-470d-8561-9ff23a87d3dc admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:15.485203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa2e5b-722b-470d-8561-9ff23a87d3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:15.485203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa2e5b-722b-470d-8561-9ff23a87d3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:15.485517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa2e5b-722b-470d-8561-9ff23a87d3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:15.485517 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa2e5b-722b-470d-8561-9ff23a87d3dc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:15.491388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1325/2672] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:15.495304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ffef0-b9db-479f-a982-2b80e395a059 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:15.507987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ffef0-b9db-479f-a982-2b80e395a059 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:15.548425 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc214f-9dce-4b57-ba0a-908261346d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:15.549486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc214f-9dce-4b57-ba0a-908261346d4e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:15.549973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc214f-9dce-4b57-ba0a-908261346d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:15.551156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc214f-9dce-4b57-ba0a-908261346d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:15.552292 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc214f-9dce-4b57-ba0a-908261346d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:15.556325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc214f-9dce-4b57-ba0a-908261346d4e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:15.569546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1348/2673] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:15.574290 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa2e5b-722b-470d-8561-9ff23a87d3dc admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:15.594347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa2e5b-722b-470d-8561-9ff23a87d3dc admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:15.604620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:15.606140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:15.606428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:15.606711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] PATH_INFO: `/v3/projects/d479435ca1724c15aac92f23856e7c93/users/bab9b88c7c4440b1b7a57077e46d2eb7/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:15.641250 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d479435ca1724c15aac92f23856e7c93, user_id=bab9b88c7c4440b1b7a57077e46d2eb7, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:15.643940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:15.741714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1326/2674] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:42:15 2022] PUT /identity/v3/projects/d479435ca1724c15aac92f23856e7c93/users/bab9b88c7c4440b1b7a57077e46d2eb7/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:15.753332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54a7c4-1bc8-4015-9970-dba8beef633e None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:15.754032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54a7c4-1bc8-4015-9970-dba8beef633e None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:15.754546 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54a7c4-1bc8-4015-9970-dba8beef633e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:15.797531 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-7f54a7c4-1bc8-4015-9970-dba8beef633e None None] MFA Rules not processed for user `bab9b88c7c4440b1b7a57077e46d2eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:15.798850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-7f54a7c4-1bc8-4015-9970-dba8beef633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:15.908927 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1349/2675] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:15.921242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:15.923297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:16.001350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68a73-3887-4b7e-a61e-21acfbf72df1 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:16.005571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:16.006318 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:16.006685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:16.007042 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:16.020819 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:16.022425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:16.033900 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1327/2676] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:16.184324 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33690f-08d6-475a-ba2d-9ddc0153f883 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:16.186359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33690f-08d6-475a-ba2d-9ddc0153f883 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:16.186359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33690f-08d6-475a-ba2d-9ddc0153f883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:16.223142 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-4d33690f-08d6-475a-ba2d-9ddc0153f883 None None] MFA Rules not processed for user `517928d85b764c089c6dd5b2d088639c`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:16.224370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4d33690f-08d6-475a-ba2d-9ddc0153f883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:16.304750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1350/2677] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:16.309555 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:16.311331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:16.390872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd997d6e-3de4-4db5-80c3-69b80c713610 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:16.395386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29d4a5-8191-4f59-b391-f4a34ece8455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:16.396081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29d4a5-8191-4f59-b391-f4a34ece8455 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:16.396368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29d4a5-8191-4f59-b391-f4a34ece8455 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:16.396683 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29d4a5-8191-4f59-b391-f4a34ece8455 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:16.397380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29d4a5-8191-4f59-b391-f4a34ece8455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:16.399281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29d4a5-8191-4f59-b391-f4a34ece8455 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:16.482050 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1328/2678] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:42:16 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:16.486594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33690f-08d6-475a-ba2d-9ddc0153f883 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:16.498311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33690f-08d6-475a-ba2d-9ddc0153f883 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:16.559657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20c2bd-1db5-4171-9263-6691e586f6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:16.560370 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20c2bd-1db5-4171-9263-6691e586f6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:16.560758 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20c2bd-1db5-4171-9263-6691e586f6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:16.561043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20c2bd-1db5-4171-9263-6691e586f6d3 admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:16.561750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20c2bd-1db5-4171-9263-6691e586f6d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:16.563878 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20c2bd-1db5-4171-9263-6691e586f6d3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:16.620885 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:16.655485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1351/2679] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:42:16 2022] POST /identity/v3/users => generated 295 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:16.659629 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29d4a5-8191-4f59-b391-f4a34ece8455 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:16.670954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29d4a5-8191-4f59-b391-f4a34ece8455 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:16.675532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3395b1e-246a-42b6-961e-beaf0089d028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:16.676275 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3395b1e-246a-42b6-961e-beaf0089d028 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:16.676586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3395b1e-246a-42b6-961e-beaf0089d028 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:16.676871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3395b1e-246a-42b6-961e-beaf0089d028 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:16.677602 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3395b1e-246a-42b6-961e-beaf0089d028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:16.679205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3395b1e-246a-42b6-961e-beaf0089d028 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:16.689309 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1329/2680] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:16.693364 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20c2bd-1db5-4171-9263-6691e586f6d3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:16.703561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20c2bd-1db5-4171-9263-6691e586f6d3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:16.708089 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4303973-a667-471d-b383-404cd25ce9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:16.709004 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303973-a667-471d-b383-404cd25ce9b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:16.709288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303973-a667-471d-b383-404cd25ce9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:16.709567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303973-a667-471d-b383-404cd25ce9b0 admin admin] PATH_INFO: `/v3/projects/462ba7aac9634b0eb45f69eeed5823ec/users/dc61a69b9a8248ac8ec5a21c95441852/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:16.726601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4303973-a667-471d-b383-404cd25ce9b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=462ba7aac9634b0eb45f69eeed5823ec, user_id=dc61a69b9a8248ac8ec5a21c95441852, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:16.727417 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4303973-a667-471d-b383-404cd25ce9b0 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:16.803987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1352/2681] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:42:16 2022] PUT /identity/v3/projects/462ba7aac9634b0eb45f69eeed5823ec/users/dc61a69b9a8248ac8ec5a21c95441852/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:16.807975 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3395b1e-246a-42b6-961e-beaf0089d028 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:16.818220 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3395b1e-246a-42b6-961e-beaf0089d028 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:16.837369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:16.877369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b564d2-5410-4928-b1ba-d7f70f8d4f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:16.878041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b564d2-5410-4928-b1ba-d7f70f8d4f29 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:16.878327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b564d2-5410-4928-b1ba-d7f70f8d4f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:16.878604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b564d2-5410-4928-b1ba-d7f70f8d4f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:16.879327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b564d2-5410-4928-b1ba-d7f70f8d4f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:16.881132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b564d2-5410-4928-b1ba-d7f70f8d4f29 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:16.891520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1330/2682] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:42:16.895642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4303973-a667-471d-b383-404cd25ce9b0 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:16.908469 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4303973-a667-471d-b383-404cd25ce9b0 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:16.913652 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7471d9-1fa5-4787-93cc-c55cd39dfe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:16.914559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7471d9-1fa5-4787-93cc-c55cd39dfe16 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:16.914850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7471d9-1fa5-4787-93cc-c55cd39dfe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:16.915181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7471d9-1fa5-4787-93cc-c55cd39dfe16 admin admin] PATH_INFO: `/v3/projects/462ba7aac9634b0eb45f69eeed5823ec/users/dc61a69b9a8248ac8ec5a21c95441852/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:16.917505 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7471d9-1fa5-4787-93cc-c55cd39dfe16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=462ba7aac9634b0eb45f69eeed5823ec, user_id=dc61a69b9a8248ac8ec5a21c95441852, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:16.920876 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7471d9-1fa5-4787-93cc-c55cd39dfe16 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:16.935454 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1353/2683] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:42:16 2022] PUT /identity/v3/projects/462ba7aac9634b0eb45f69eeed5823ec/users/dc61a69b9a8248ac8ec5a21c95441852/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:42:20.704794 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b564d2-5410-4928-b1ba-d7f70f8d4f29 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:20.716221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b564d2-5410-4928-b1ba-d7f70f8d4f29 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:20.784943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:20.785962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:20.786325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:20.786802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] PATH_INFO: `/v3/users/bab9b88c7c4440b1b7a57077e46d2eb7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:20.788363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bab9b88c7c4440b1b7a57077e46d2eb7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:20.790879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:21.035848 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bab9b88c7c4440b1b7a57077e46d2eb7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:21.045815 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bab9b88c7c4440b1b7a57077e46d2eb7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:21.046911 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bab9b88c7c4440b1b7a57077e46d2eb7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:21.047805 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bab9b88c7c4440b1b7a57077e46d2eb7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:21.127051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bab9b88c7c4440b1b7a57077e46d2eb7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:21.156611 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1331/2684] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:20 2022] DELETE /identity/v3/users/bab9b88c7c4440b1b7a57077e46d2eb7 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:21.160153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac3440-f685-4fd1-88d8-1450b64238bd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:21.161139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac3440-f685-4fd1-88d8-1450b64238bd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:21.162159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac3440-f685-4fd1-88d8-1450b64238bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:21.186240 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-64ac3440-f685-4fd1-88d8-1450b64238bd None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:21.187485 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-64ac3440-f685-4fd1-88d8-1450b64238bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:21.289002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1354/2685] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:21.300329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:21.302370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:21.365187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:21.398920 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158db2b-568d-497b-8488-2d03217471fa admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:21.408772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eb998-1065-4523-9560-5e2df5a82f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:21.409717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb998-1065-4523-9560-5e2df5a82f8e service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:21.410009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb998-1065-4523-9560-5e2df5a82f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:21.410289 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb998-1065-4523-9560-5e2df5a82f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:21.427600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eb998-1065-4523-9560-5e2df5a82f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:21.429929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eb998-1065-4523-9560-5e2df5a82f8e service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:21.444779 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1332/2686] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:22.045231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac3440-f685-4fd1-88d8-1450b64238bd None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:22.045231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-64ac3440-f685-4fd1-88d8-1450b64238bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:22.055738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac3440-f685-4fd1-88d8-1450b64238bd None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:22.064747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:22.065938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:22.066340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:22.066682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] PATH_INFO: `/v3/projects/d479435ca1724c15aac92f23856e7c93` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:22.068427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d479435ca1724c15aac92f23856e7c93)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:22.072567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:22.207696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd479435ca1724c15aac92f23856e7c93', 'request_id': 'req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:22.276441 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd479435ca1724c15aac92f23856e7c93', 'request_id': 'req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:22.290581 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd479435ca1724c15aac92f23856e7c93', 'request_id': 'req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:22.326591 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] The token cache is being invalidate because project d479435ca1724c15aac92f23856e7c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:22.327363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:22.328354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d8a8c451-1cca-4eff-b3a7-44066ef9bf9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:22.330284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1355/2687] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:22 2022] DELETE /identity/v3/projects/d479435ca1724c15aac92f23856e7c93 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:22.335233 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ad86b-bbbe-46e8-a8e3-bc1b4d448e60 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:22.335774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ad86b-bbbe-46e8-a8e3-bc1b4d448e60 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:22.335774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ad86b-bbbe-46e8-a8e3-bc1b4d448e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:22.371585 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-aa8ad86b-bbbe-46e8-a8e3-bc1b4d448e60 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:22.372840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-aa8ad86b-bbbe-46e8-a8e3-bc1b4d448e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:22.484248 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1333/2688] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:22.494270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c72cd4-3b58-4e2c-b00c-298b166bc8a4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:22.495575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c72cd4-3b58-4e2c-b00c-298b166bc8a4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:22.495575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c72cd4-3b58-4e2c-b00c-298b166bc8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:22.517495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-42c72cd4-3b58-4e2c-b00c-298b166bc8a4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:22.518888 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-42c72cd4-3b58-4e2c-b00c-298b166bc8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:22.529613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1356/2689] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:22.537297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ad86b-bbbe-46e8-a8e3-bc1b4d448e60 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:22.552179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ad86b-bbbe-46e8-a8e3-bc1b4d448e60 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:22.553772 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:22.554579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:22.554964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:22.555342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:22.556151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:42:22.556659 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:22.558899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:22.580572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1334/2690] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:42:22.585830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c72cd4-3b58-4e2c-b00c-298b166bc8a4 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:22.604408 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c72cd4-3b58-4e2c-b00c-298b166bc8a4 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:22.623811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa5f94-5394-4b24-842f-f5c906e1614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:22.627492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa5f94-5394-4b24-842f-f5c906e1614c admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:22.627845 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa5f94-5394-4b24-842f-f5c906e1614c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:22.628561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa5f94-5394-4b24-842f-f5c906e1614c admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:22.629592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa5f94-5394-4b24-842f-f5c906e1614c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:22.632059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa5f94-5394-4b24-842f-f5c906e1614c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:22.651432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:22.782294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1357/2691] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:42:22 2022] POST /identity/v3/projects => generated 399 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:22.788099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:22.798831 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8ec07-21db-4225-9586-c03b9b4e289b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:22.864415 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322bbab-771c-4279-bd66-fd5c3782573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:22.865426 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322bbab-771c-4279-bd66-fd5c3782573c admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:22.865633 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322bbab-771c-4279-bd66-fd5c3782573c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:22.865905 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322bbab-771c-4279-bd66-fd5c3782573c admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:22.866619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322bbab-771c-4279-bd66-fd5c3782573c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:22.869070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322bbab-771c-4279-bd66-fd5c3782573c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:22.962849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1335/2692] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:42:22 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:22.967791 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa5f94-5394-4b24-842f-f5c906e1614c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:22.981691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa5f94-5394-4b24-842f-f5c906e1614c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:22.986780 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002de5d-0a1d-497e-91f2-6f1cc82bd948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:22.987820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002de5d-0a1d-497e-91f2-6f1cc82bd948 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:22.987820 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002de5d-0a1d-497e-91f2-6f1cc82bd948 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:22.988021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002de5d-0a1d-497e-91f2-6f1cc82bd948 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:22.989239 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002de5d-0a1d-497e-91f2-6f1cc82bd948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:22.990905 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002de5d-0a1d-497e-91f2-6f1cc82bd948 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:23.003808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1358/2693] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:23.007903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322bbab-771c-4279-bd66-fd5c3782573c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:23.044034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322bbab-771c-4279-bd66-fd5c3782573c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:23.049116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1e0de-a11b-449d-8c9f-ad2c8ebdb324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:23.049798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1e0de-a11b-449d-8c9f-ad2c8ebdb324 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:23.050087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1e0de-a11b-449d-8c9f-ad2c8ebdb324 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:23.050374 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1e0de-a11b-449d-8c9f-ad2c8ebdb324 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:23.051140 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1e0de-a11b-449d-8c9f-ad2c8ebdb324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:23.052822 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1e0de-a11b-449d-8c9f-ad2c8ebdb324 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:23.071745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1336/2694] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:23.073077 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002de5d-0a1d-497e-91f2-6f1cc82bd948 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:23.092320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002de5d-0a1d-497e-91f2-6f1cc82bd948 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:23.099479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:23.099965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:23.100256 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:23.100558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] PATH_INFO: `/v3/projects/392a865f6ca84f279bf17e16b0362ec3/users/918cf82634484ef4a57aa18cd2263c54/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:23.127754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=392a865f6ca84f279bf17e16b0362ec3, user_id=918cf82634484ef4a57aa18cd2263c54, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:23.130650 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:23.212157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1359/2695] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:42:23 2022] PUT /identity/v3/projects/392a865f6ca84f279bf17e16b0362ec3/users/918cf82634484ef4a57aa18cd2263c54/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:23.223013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae364d-d31b-488c-a810-0ac77a212ee7 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:23.223448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae364d-d31b-488c-a810-0ac77a212ee7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:23.223646 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae364d-d31b-488c-a810-0ac77a212ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:23.259798 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a5ae364d-d31b-488c-a810-0ac77a212ee7 None None] MFA Rules not processed for user `918cf82634484ef4a57aa18cd2263c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:23.261090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a5ae364d-d31b-488c-a810-0ac77a212ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:23.309036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:23.354391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1337/2696] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:23.366383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:23.368108 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:23.371428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73d28-3de9-4cae-ab0a-e4d6c4fab49c None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:23.376682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73d28-3de9-4cae-ab0a-e4d6c4fab49c None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:23.376682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73d28-3de9-4cae-ab0a-e4d6c4fab49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:23.399874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9df73d28-3de9-4cae-ab0a-e4d6c4fab49c None None] MFA Rules not processed for user `dc61a69b9a8248ac8ec5a21c95441852`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:23.401169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9df73d28-3de9-4cae-ab0a-e4d6c4fab49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:23.438862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d9e06-b7a0-4742-a205-373edb26c20d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:23.443785 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:23.444559 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:23.444958 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:23.445312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:23.455198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:23.457157 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:23.467774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1360/2697] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:23.485329 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1338/2698] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:42:23.500411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:23.501941 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:23.595747 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1d1ab-3777-48de-a592-0111f726261b service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:23.601614 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:23.602430 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:23.602727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:23.603014 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:23.604953 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:23.677938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:23.679554 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:23.684833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:23.692858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1361/2699] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:23.780736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df73d28-3de9-4cae-ab0a-e4d6c4fab49c None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:23.782170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9df73d28-3de9-4cae-ab0a-e4d6c4fab49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:23.842384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df73d28-3de9-4cae-ab0a-e4d6c4fab49c None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:23.846126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:23.846915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:23.847204 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:23.847515 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:23.849392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:23.953663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:23.955230 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:23.966924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1339/2700] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:28.480484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:28.482064 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:28.497033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e74dd-bdf4-43ff-9be5-a70ea0f643b9 service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:28.502141 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:28.503020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:28.503300 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:28.503763 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] PATH_INFO: `/v3/users/31973e5a04c14cfc90ab377eac5f4194` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:28.505276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31973e5a04c14cfc90ab377eac5f4194)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:28.507280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:28.693968 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31973e5a04c14cfc90ab377eac5f4194'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:28.706884 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31973e5a04c14cfc90ab377eac5f4194'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:28.713859 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31973e5a04c14cfc90ab377eac5f4194'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:28.714522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31973e5a04c14cfc90ab377eac5f4194'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:28.715123 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-2e60b498-b8c5-400c-8e50-00eb65ed1fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31973e5a04c14cfc90ab377eac5f4194'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:28.788673 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1362/2701] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:28 2022] DELETE /identity/v3/users/31973e5a04c14cfc90ab377eac5f4194 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:29.016407 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:29.029139 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740555b-ebe2-4709-a95d-f4cafce1ce1c service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:29.037872 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=100 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:29.089635 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:29.090487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.090826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.091116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] PATH_INFO: `/v3/projects/494ce289f1774df0833301de8e46ae11` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.092579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=494ce289f1774df0833301de8e46ae11)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:29.094817 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:29.218211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494ce289f1774df0833301de8e46ae11', 'request_id': 'req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:29.292039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494ce289f1774df0833301de8e46ae11', 'request_id': 'req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:29.299256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494ce289f1774df0833301de8e46ae11', 'request_id': 'req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:29.311874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] The token cache is being invalidate because project 494ce289f1774df0833301de8e46ae11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:29.312085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:29.312773 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-9fb155b5-ad01-4040-8cee-3d5d1b1d0c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:29.314331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1340/2702] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:29 2022] DELETE /identity/v3/projects/494ce289f1774df0833301de8e46ae11 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:29.319564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9ba4-3fe2-4007-a81d-75128012d7d4 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.319904 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9ba4-3fe2-4007-a81d-75128012d7d4 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.320228 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9ba4-3fe2-4007-a81d-75128012d7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.341308 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-cb3c9ba4-3fe2-4007-a81d-75128012d7d4 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:29.342560 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-cb3c9ba4-3fe2-4007-a81d-75128012d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:29.412392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1363/2703] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:29.421194 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc882ba-59b5-4e4c-827e-5fdc239c0bbc None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.421508 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc882ba-59b5-4e4c-827e-5fdc239c0bbc None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.421789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc882ba-59b5-4e4c-827e-5fdc239c0bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.440417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4fc882ba-59b5-4e4c-827e-5fdc239c0bbc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:29.441661 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4fc882ba-59b5-4e4c-827e-5fdc239c0bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:29.450739 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1341/2704] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:29.455336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c9ba4-3fe2-4007-a81d-75128012d7d4 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:29.465794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c9ba4-3fe2-4007-a81d-75128012d7d4 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:29.470627 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:29.471291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.471576 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.471887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.472713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:42:29.473022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:29.474617 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:29.488038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1364/2705] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:42:29.492331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc882ba-59b5-4e4c-827e-5fdc239c0bbc None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:29.502323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc882ba-59b5-4e4c-827e-5fdc239c0bbc None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:29.508038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b8826-f987-4789-a29e-fd102c570dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:29.508948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8826-f987-4789-a29e-fd102c570dfe admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.509246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8826-f987-4789-a29e-fd102c570dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.509529 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8826-f987-4789-a29e-fd102c570dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.510279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b8826-f987-4789-a29e-fd102c570dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:29.514979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b8826-f987-4789-a29e-fd102c570dfe admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:29.623070 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1342/2706] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:42:29 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:29.627193 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:29.638669 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37de03-f32a-4217-8b75-0814e7ebcc5e admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:29.690713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:29.698344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b737b6-3732-4851-ba53-9f27183b059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:29.699638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b737b6-3732-4851-ba53-9f27183b059b admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.699950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b737b6-3732-4851-ba53-9f27183b059b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.700231 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b737b6-3732-4851-ba53-9f27183b059b admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.700957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b737b6-3732-4851-ba53-9f27183b059b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:29.702952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b737b6-3732-4851-ba53-9f27183b059b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:29.792978 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1365/2707] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:42:29 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:42:29.797123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b8826-f987-4789-a29e-fd102c570dfe admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:29.807984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b8826-f987-4789-a29e-fd102c570dfe admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:29.813369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103647d-691f-4bd5-a31f-a46b4d63c584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:29.814095 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103647d-691f-4bd5-a31f-a46b4d63c584 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.814388 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103647d-691f-4bd5-a31f-a46b4d63c584 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.814673 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103647d-691f-4bd5-a31f-a46b4d63c584 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.815408 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103647d-691f-4bd5-a31f-a46b4d63c584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:29.817048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103647d-691f-4bd5-a31f-a46b4d63c584 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:29.827826 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1343/2708] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:29.832185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b737b6-3732-4851-ba53-9f27183b059b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:29.842461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b737b6-3732-4851-ba53-9f27183b059b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:29.848005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26028aa4-1125-4e39-a2ac-4429c7df75f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:29.848782 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26028aa4-1125-4e39-a2ac-4429c7df75f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.849110 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26028aa4-1125-4e39-a2ac-4429c7df75f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.849431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26028aa4-1125-4e39-a2ac-4429c7df75f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.850201 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26028aa4-1125-4e39-a2ac-4429c7df75f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:29.853205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26028aa4-1125-4e39-a2ac-4429c7df75f4 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:29.873822 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1366/2709] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:42:29.878018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103647d-691f-4bd5-a31f-a46b4d63c584 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:29.889738 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103647d-691f-4bd5-a31f-a46b4d63c584 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:29.894671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:29.895561 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:29.895847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:29.896126 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] PATH_INFO: `/v3/projects/f8eb9ad6ea03410f9956bbabeddcd5b2/users/b3e4c68725cb4a9ea9024c3205dd5588/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:29.911282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8eb9ad6ea03410f9956bbabeddcd5b2, user_id=b3e4c68725cb4a9ea9024c3205dd5588, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:29.914009 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:30.003914 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1344/2710] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:42:29 2022] PUT /identity/v3/projects/f8eb9ad6ea03410f9956bbabeddcd5b2/users/b3e4c68725cb4a9ea9024c3205dd5588/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:30.009151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4f4df-b493-4f4f-8d58-195f6af85721 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:30.009455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4f4df-b493-4f4f-8d58-195f6af85721 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:30.009739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4f4df-b493-4f4f-8d58-195f6af85721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:30.030490 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-aaa4f4df-b493-4f4f-8d58-195f6af85721 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:30.031722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-aaa4f4df-b493-4f4f-8d58-195f6af85721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:30.094959 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1367/2711] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:30.104996 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:30.106750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:30.149859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:30.165455 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ecaee-92e8-49f5-b361-c96d341e1ac6 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:30.169350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca8cbc-9bda-45b1-bb90-447eec278120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:30.170007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8cbc-9bda-45b1-bb90-447eec278120 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:30.170296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8cbc-9bda-45b1-bb90-447eec278120 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:30.170580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8cbc-9bda-45b1-bb90-447eec278120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:30.180090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca8cbc-9bda-45b1-bb90-447eec278120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:30.181578 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca8cbc-9bda-45b1-bb90-447eec278120 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:30.194829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1345/2712] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:32.774478 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4f4df-b493-4f4f-8d58-195f6af85721 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:32.775899 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-aaa4f4df-b493-4f4f-8d58-195f6af85721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:32.789113 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4f4df-b493-4f4f-8d58-195f6af85721 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:32.796042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:32.796901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:32.797187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:32.797467 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] PATH_INFO: `/v3/users/918cf82634484ef4a57aa18cd2263c54` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:32.798684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=918cf82634484ef4a57aa18cd2263c54)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:32.800684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:33.036479 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918cf82634484ef4a57aa18cd2263c54'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:33.057411 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918cf82634484ef4a57aa18cd2263c54'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:33.068082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918cf82634484ef4a57aa18cd2263c54'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:33.069663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918cf82634484ef4a57aa18cd2263c54'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:33.070442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918cf82634484ef4a57aa18cd2263c54'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:33.153594 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1368/2713] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:32 2022] DELETE /identity/v3/users/918cf82634484ef4a57aa18cd2263c54 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:33.159405 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5564-59af-4845-99ae-2a51926f408b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:33.159870 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5564-59af-4845-99ae-2a51926f408b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:33.159977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5564-59af-4845-99ae-2a51926f408b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:33.184855 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-f39d5564-59af-4845-99ae-2a51926f408b None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:33.186130 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f39d5564-59af-4845-99ae-2a51926f408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:33.283705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1346/2714] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:33.303889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:33.303889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:33.380730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:33.385162 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3742c-887d-493e-85ed-ad542109be47 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:33.389841 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6c715-f0cc-4faf-887c-3392b666d6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:33.390524 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6c715-f0cc-4faf-887c-3392b666d6bf service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:33.390816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6c715-f0cc-4faf-887c-3392b666d6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:33.391353 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6c715-f0cc-4faf-887c-3392b666d6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:33.406920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6c715-f0cc-4faf-887c-3392b666d6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:33.408694 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6c715-f0cc-4faf-887c-3392b666d6bf service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:33.437303 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1369/2715] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:33.992943 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d5564-59af-4845-99ae-2a51926f408b None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:33.994707 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-f39d5564-59af-4845-99ae-2a51926f408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:34.020596 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d5564-59af-4845-99ae-2a51926f408b None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:34.028026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:34.028967 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:34.031424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:34.031424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] PATH_INFO: `/v3/projects/392a865f6ca84f279bf17e16b0362ec3` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:34.031424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392a865f6ca84f279bf17e16b0362ec3)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:34.032799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:34.168242 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392a865f6ca84f279bf17e16b0362ec3', 'request_id': 'req-bbc02d62-ad3d-4abb-8e58-72c25423fd92'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:34.244421 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392a865f6ca84f279bf17e16b0362ec3', 'request_id': 'req-bbc02d62-ad3d-4abb-8e58-72c25423fd92'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:34.251030 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392a865f6ca84f279bf17e16b0362ec3', 'request_id': 'req-bbc02d62-ad3d-4abb-8e58-72c25423fd92'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:34.265496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] The token cache is being invalidate because project 392a865f6ca84f279bf17e16b0362ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:34.265876 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:34.266774 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:34.268966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1347/2716] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:33 2022] DELETE /identity/v3/projects/392a865f6ca84f279bf17e16b0362ec3 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:35.935242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f35b5-bd22-43b5-a1bb-86547c42a564 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:35.935743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f35b5-bd22-43b5-a1bb-86547c42a564 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:35.935846 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f35b5-bd22-43b5-a1bb-86547c42a564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:35.975076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b62f35b5-bd22-43b5-a1bb-86547c42a564 None None] MFA Rules not processed for user `b3e4c68725cb4a9ea9024c3205dd5588`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:35.975076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b62f35b5-bd22-43b5-a1bb-86547c42a564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:36.064667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1370/2717] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:36.082918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:36.084413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:36.163138 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:36.169751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc02d62-ad3d-4abb-8e58-72c25423fd92 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:36.175302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c6530-7637-415a-a500-00e8886a0963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:36.176038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6530-7637-415a-a500-00e8886a0963 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:36.176396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6530-7637-415a-a500-00e8886a0963 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:36.176709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6530-7637-415a-a500-00e8886a0963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:36.187023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c6530-7637-415a-a500-00e8886a0963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:36.188701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c6530-7637-415a-a500-00e8886a0963 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:36.199598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1348/2718] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:39.534279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610902e0-b409-4605-be6c-1973af73be47 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:39.534842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610902e0-b409-4605-be6c-1973af73be47 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:39.535079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610902e0-b409-4605-be6c-1973af73be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:39.555739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-610902e0-b409-4605-be6c-1973af73be47 None None] MFA Rules not processed for user `b3e4c68725cb4a9ea9024c3205dd5588`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:39.557080 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-610902e0-b409-4605-be6c-1973af73be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:39.565910 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1371/2719] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:44.031465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d9b87-2334-43be-8023-ef7f79beb589 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:44.031954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d9b87-2334-43be-8023-ef7f79beb589 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:44.033149 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d9b87-2334-43be-8023-ef7f79beb589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:44.053568 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-709d9b87-2334-43be-8023-ef7f79beb589 None None] MFA Rules not processed for user `517928d85b764c089c6dd5b2d088639c`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:44.054767 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-709d9b87-2334-43be-8023-ef7f79beb589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:44.107937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1349/2720] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:42:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:48.145453 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23466ef8-adea-40ef-b9b6-d9c998b7ce8a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:48.145920 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23466ef8-adea-40ef-b9b6-d9c998b7ce8a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:48.146020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23466ef8-adea-40ef-b9b6-d9c998b7ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:48.163367 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-23466ef8-adea-40ef-b9b6-d9c998b7ce8a None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:42:48.164602 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-23466ef8-adea-40ef-b9b6-d9c998b7ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:48.219185 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1372/2721] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jun 16 00:42:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:42:48.355169 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d9b87-2334-43be-8023-ef7f79beb589 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:48.356618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-709d9b87-2334-43be-8023-ef7f79beb589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:48.426069 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d9b87-2334-43be-8023-ef7f79beb589 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:48.431218 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134132c-da37-44f2-a90f-24bbc6385080 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:48.436362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134132c-da37-44f2-a90f-24bbc6385080 service swift] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:48.436362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134132c-da37-44f2-a90f-24bbc6385080 service swift] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:48.436362 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134132c-da37-44f2-a90f-24bbc6385080 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:48.441829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134132c-da37-44f2-a90f-24bbc6385080 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:48.443442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134132c-da37-44f2-a90f-24bbc6385080 service swift] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:48.452174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1350/2722] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:42:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:42:57.664012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23466ef8-adea-40ef-b9b6-d9c998b7ce8a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:57.665574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-23466ef8-adea-40ef-b9b6-d9c998b7ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:57.732432 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23466ef8-adea-40ef-b9b6-d9c998b7ce8a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:57.736895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:57.737765 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:57.738054 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:57.738336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] PATH_INFO: `/v3/users/f2aa1b8204b1494589e096ad775ce4d9` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:57.739533 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2aa1b8204b1494589e096ad775ce4d9)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:57.741494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:57.763936 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:57.946477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2aa1b8204b1494589e096ad775ce4d9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:57.960241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2aa1b8204b1494589e096ad775ce4d9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:57.967283 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2aa1b8204b1494589e096ad775ce4d9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:57.968514 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2aa1b8204b1494589e096ad775ce4d9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:57.969151 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2aa1b8204b1494589e096ad775ce4d9'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.032252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1373/2723] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:57 2022] DELETE /identity/v3/users/f2aa1b8204b1494589e096ad775ce4d9 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:58.273864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134132c-da37-44f2-a90f-24bbc6385080 service swift] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:58.286244 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134132c-da37-44f2-a90f-24bbc6385080 service swift] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:58.350215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:58.351108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:58.351419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:58.351702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] PATH_INFO: `/v3/projects/ad47d922a30a4c55a781c52c5bd5236f` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:58.353007 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad47d922a30a4c55a781c52c5bd5236f)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:58.354940 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:58.450229 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:58.466325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad47d922a30a4c55a781c52c5bd5236f', 'request_id': 'req-c59f8374-cab9-41b6-bbcf-69897497e091'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.602733 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad47d922a30a4c55a781c52c5bd5236f', 'request_id': 'req-c59f8374-cab9-41b6-bbcf-69897497e091'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.608574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad47d922a30a4c55a781c52c5bd5236f', 'request_id': 'req-c59f8374-cab9-41b6-bbcf-69897497e091'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.619526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] The token cache is being invalidate because project ad47d922a30a4c55a781c52c5bd5236f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:58.619814 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.620433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.622054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1351/2724] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:58 2022] DELETE /identity/v3/projects/ad47d922a30a4c55a781c52c5bd5236f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:58.626687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:58.628079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:58.641493 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73ba62-9dd9-4e23-9bdd-e13190aaf7bd admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:58.696512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:58.697407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:58.697690 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:58.697964 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] PATH_INFO: `/v3/users/4bf1fbaebebf4d5cb4c24a69dd123261` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:58.699177 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bf1fbaebebf4d5cb4c24a69dd123261)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:58.701216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:58.910585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bf1fbaebebf4d5cb4c24a69dd123261'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.923034 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bf1fbaebebf4d5cb4c24a69dd123261'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.929890 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bf1fbaebebf4d5cb4c24a69dd123261'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.930558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bf1fbaebebf4d5cb4c24a69dd123261'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.931195 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bf1fbaebebf4d5cb4c24a69dd123261'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:58.988834 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1374/2725] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:58 2022] DELETE /identity/v3/users/4bf1fbaebebf4d5cb4c24a69dd123261 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:59.232544 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:59.235759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:59.247091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8374-cab9-41b6-bbcf-69897497e091 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:59.303399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:59.304322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:59.304636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:59.304918 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] PATH_INFO: `/v3/projects/d9eec0a2c24d4981997c685919bebeaa` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:59.306096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9eec0a2c24d4981997c685919bebeaa)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:59.307976 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:59.443959 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9eec0a2c24d4981997c685919bebeaa', 'request_id': 'req-2bee8a50-5928-465c-b9f4-667f49612a69'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:59.494207 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9eec0a2c24d4981997c685919bebeaa', 'request_id': 'req-2bee8a50-5928-465c-b9f4-667f49612a69'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:59.500879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9eec0a2c24d4981997c685919bebeaa', 'request_id': 'req-2bee8a50-5928-465c-b9f4-667f49612a69'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:59.512896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] The token cache is being invalidate because project d9eec0a2c24d4981997c685919bebeaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:59.513189 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:59.513859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:59.516137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1352/2726] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:59 2022] DELETE /identity/v3/projects/d9eec0a2c24d4981997c685919bebeaa => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:43:06.877452 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:43:06.878913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:43:06.899295 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:43:06.935049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7dae7-c35d-431b-8639-b0723724bb69 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:43:06.982954 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:43:06.983630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:43:06.983918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:43:06.984199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:43:06.985998 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:43:07.047056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:43:07.048482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:43:07.057909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1375/2727] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:43:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:43:09.509631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:43:17.053689 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=62 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:43:48.830261 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:43:48.831676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:43:48.900068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee8a50-5928-465c-b9f4-667f49612a69 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:43:48.905060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:43:48.905709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:43:48.906001 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:43:48.906282 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:43:48.908079 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:43:48.919384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:43:48.920979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:43:48.930791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1353/2728] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:43:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:43:58.926547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=32 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:44:34.722056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:44:34.723492 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:44:34.734176 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657e248-4c93-40ad-ac7d-f2987333d8fb service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:44:34.740754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268d569-06c3-483a-bbf9-cec66d1d54e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:44:34.740754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d569-06c3-483a-bbf9-cec66d1d54e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:44:34.740754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d569-06c3-483a-bbf9-cec66d1d54e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:44:34.740754 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d569-06c3-483a-bbf9-cec66d1d54e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:44:34.741153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1268d569-06c3-483a-bbf9-cec66d1d54e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:44:34.810946 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268d569-06c3-483a-bbf9-cec66d1d54e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:44:34.812373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268d569-06c3-483a-bbf9-cec66d1d54e2 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:44:34.824065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1376/2729] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:44:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:44:35.996919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:44:35.998323 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:44:36.009496 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31db40-d8cd-4082-8806-d955c8cba22b service placement] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:44:36.013059 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:44:36.013685 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:44:36.013965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:44:36.014238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:44:36.015901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:44:36.028547 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:44:36.029956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:44:36.041317 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1354/2730] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:44:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:44:44.818975 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=34 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:44:46.036512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=6 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:45:09.220586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6b669-8834-42ae-8285-1455f05e1005 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:45:09.221122 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6b669-8834-42ae-8285-1455f05e1005 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:45:09.221217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6b669-8834-42ae-8285-1455f05e1005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:45:09.239158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-34e6b669-8834-42ae-8285-1455f05e1005 None None] MFA Rules not processed for user `517928d85b764c089c6dd5b2d088639c`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:45:09.240360 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-34e6b669-8834-42ae-8285-1455f05e1005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:45:09.293670 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1377/2731] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:45:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:45:13.183852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:45:13.192562 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f81d-c817-4864-a9a2-f9a3d805a8ea service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:45:13.196588 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5574c-bd7f-4191-bb39-b20871dc9341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:45:13.201580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5574c-bd7f-4191-bb39-b20871dc9341 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:45:13.201580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5574c-bd7f-4191-bb39-b20871dc9341 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:45:13.201580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5574c-bd7f-4191-bb39-b20871dc9341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:45:13.201580 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3cd5574c-bd7f-4191-bb39-b20871dc9341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:45:13.210241 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5574c-bd7f-4191-bb39-b20871dc9341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:45:13.211653 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5574c-bd7f-4191-bb39-b20871dc9341 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:45:13.219737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1355/2732] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:45:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:45:19.287215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=25 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:45:23.216091 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=6 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:45:24.049558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6b669-8834-42ae-8285-1455f05e1005 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:45:24.053223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-34e6b669-8834-42ae-8285-1455f05e1005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:45:24.079852 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6b669-8834-42ae-8285-1455f05e1005 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:45:24.085861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:45:24.086522 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:45:24.086809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:45:24.087094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:45:24.089012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:45:24.103530 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:45:24.105322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:45:24.124126 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1378/2733] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:45:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:45:28.600281 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7b7eb-dfdf-4866-a6f3-389150d51c90 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:45:28.600782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7b7eb-dfdf-4866-a6f3-389150d51c90 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:45:28.601490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7b7eb-dfdf-4866-a6f3-389150d51c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:45:28.620015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-d0f7b7eb-dfdf-4866-a6f3-389150d51c90 None None] MFA Rules not processed for user `b3e4c68725cb4a9ea9024c3205dd5588`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:45:28.621246 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d0f7b7eb-dfdf-4866-a6f3-389150d51c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:45:28.680308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1356/2734] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:45:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:45:34.118242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=6 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:45:38.670203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=25 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:46:53.733048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:53.734585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:46:53.749456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f7c81-7146-45ba-8c0b-b64a70c16fdf service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:53.754521 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:53.755170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:53.755455 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:53.755734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:53.757552 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:46:53.770346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:53.771943 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:53.783539 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1379/2735] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:46:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:46:54.167122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7b7eb-dfdf-4866-a6f3-389150d51c90 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:54.168606 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d0f7b7eb-dfdf-4866-a6f3-389150d51c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:46:54.233188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7b7eb-dfdf-4866-a6f3-389150d51c90 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:54.237723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:54.238540 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:54.238829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:54.239111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] PATH_INFO: `/v3/users/b3e4c68725cb4a9ea9024c3205dd5588` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:54.240296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e4c68725cb4a9ea9024c3205dd5588)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:54.242245 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:54.380657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e4c68725cb4a9ea9024c3205dd5588'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.387481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e4c68725cb4a9ea9024c3205dd5588'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.388129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e4c68725cb4a9ea9024c3205dd5588'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.388853 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e4c68725cb4a9ea9024c3205dd5588'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.442263 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-07a9d637-ad75-450a-97b1-555b34658e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e4c68725cb4a9ea9024c3205dd5588'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.460649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1357/2736] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:46:54 2022] DELETE /identity/v3/users/b3e4c68725cb4a9ea9024c3205dd5588 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:46:54.717022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:54.727676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ca6c8-c6a6-4844-a377-c2e352819884 service placement] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:54.787974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:54.788868 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:54.789158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:54.789437 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] PATH_INFO: `/v3/projects/f8eb9ad6ea03410f9956bbabeddcd5b2` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:54.790625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8eb9ad6ea03410f9956bbabeddcd5b2)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:54.793192 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:54.916856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8eb9ad6ea03410f9956bbabeddcd5b2', 'request_id': 'req-d3820f31-ba3b-451b-ae8f-b35f252289eb'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.972934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8eb9ad6ea03410f9956bbabeddcd5b2', 'request_id': 'req-d3820f31-ba3b-451b-ae8f-b35f252289eb'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.979528 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8eb9ad6ea03410f9956bbabeddcd5b2', 'request_id': 'req-d3820f31-ba3b-451b-ae8f-b35f252289eb'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.990930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] The token cache is being invalidate because project f8eb9ad6ea03410f9956bbabeddcd5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:46:54.991216 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.991902 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d3820f31-ba3b-451b-ae8f-b35f252289eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:46:54.993663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1380/2737] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:46:54 2022] DELETE /identity/v3/projects/f8eb9ad6ea03410f9956bbabeddcd5b2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:46:54.998972 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871e4f5-90a3-439d-bc7f-9a7aec8ef0e7 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:54.999265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871e4f5-90a3-439d-bc7f-9a7aec8ef0e7 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:54.999548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871e4f5-90a3-439d-bc7f-9a7aec8ef0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.018222 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-c871e4f5-90a3-439d-bc7f-9a7aec8ef0e7 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:46:55.020136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c871e4f5-90a3-439d-bc7f-9a7aec8ef0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:46:55.079921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1358/2738] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:46:55.087939 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7c217-c7b8-4a6a-8a01-7e7fabd4a422 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.088235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7c217-c7b8-4a6a-8a01-7e7fabd4a422 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.088675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7c217-c7b8-4a6a-8a01-7e7fabd4a422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.107199 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c4e7c217-c7b8-4a6a-8a01-7e7fabd4a422 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:46:55.108391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c4e7c217-c7b8-4a6a-8a01-7e7fabd4a422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:46:55.116320 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1381/2739] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:46:55.121802 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871e4f5-90a3-439d-bc7f-9a7aec8ef0e7 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:55.131386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871e4f5-90a3-439d-bc7f-9a7aec8ef0e7 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:55.136153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:55.136828 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.137136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.137414 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.138135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:46:55.138440 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:55.140013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:55.151356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1359/2740] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:46:55.155585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7c217-c7b8-4a6a-8a01-7e7fabd4a422 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:55.164748 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7c217-c7b8-4a6a-8a01-7e7fabd4a422 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:55.169880 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32e072-b936-4ff7-aed1-26b6f1a85441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:55.170523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32e072-b936-4ff7-aed1-26b6f1a85441 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.170808 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32e072-b936-4ff7-aed1-26b6f1a85441 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.171085 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32e072-b936-4ff7-aed1-26b6f1a85441 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.171823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32e072-b936-4ff7-aed1-26b6f1a85441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:55.173737 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32e072-b936-4ff7-aed1-26b6f1a85441 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:55.249255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1382/2741] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:46:55 2022] POST /identity/v3/projects => generated 399 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:46:55.253178 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:55.262995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe838f-e8d3-4a38-a3ee-fbd65d59512f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:55.266557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:46:55.319795 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b353b9-2e58-4760-bf2d-71e8f509eed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:55.320462 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b353b9-2e58-4760-bf2d-71e8f509eed4 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.320783 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b353b9-2e58-4760-bf2d-71e8f509eed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.321065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b353b9-2e58-4760-bf2d-71e8f509eed4 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.321762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b353b9-2e58-4760-bf2d-71e8f509eed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:55.323689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b353b9-2e58-4760-bf2d-71e8f509eed4 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:55.441184 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1360/2742] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:46:55 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:46:55.445358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32e072-b936-4ff7-aed1-26b6f1a85441 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:55.456025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32e072-b936-4ff7-aed1-26b6f1a85441 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:55.460686 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97331c58-93a8-4cad-a479-1190d7843268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:55.461339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97331c58-93a8-4cad-a479-1190d7843268 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.461623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97331c58-93a8-4cad-a479-1190d7843268 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.461901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97331c58-93a8-4cad-a479-1190d7843268 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.462630 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97331c58-93a8-4cad-a479-1190d7843268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:55.464229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97331c58-93a8-4cad-a479-1190d7843268 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:55.474489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1383/2743] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:46:55.478901 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b353b9-2e58-4760-bf2d-71e8f509eed4 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:55.488619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b353b9-2e58-4760-bf2d-71e8f509eed4 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:55.493112 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e049e7-e877-4054-a68c-334034a9be3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:55.493762 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e049e7-e877-4054-a68c-334034a9be3d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.494714 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e049e7-e877-4054-a68c-334034a9be3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.494997 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e049e7-e877-4054-a68c-334034a9be3d admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.495723 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e049e7-e877-4054-a68c-334034a9be3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:55.497350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e049e7-e877-4054-a68c-334034a9be3d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:55.506956 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1361/2744] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:46:55.510862 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97331c58-93a8-4cad-a479-1190d7843268 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:55.520615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97331c58-93a8-4cad-a479-1190d7843268 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:55.525609 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695f70-e102-4946-be77-f6b3684ad440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:55.526491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695f70-e102-4946-be77-f6b3684ad440 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.526778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695f70-e102-4946-be77-f6b3684ad440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.527059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695f70-e102-4946-be77-f6b3684ad440 admin admin] PATH_INFO: `/v3/projects/c59f48b88ad24399bd4143cbec9a4eee/users/72a03b9c60724a88b42e4b27bdfe0dab/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.541777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2695f70-e102-4946-be77-f6b3684ad440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59f48b88ad24399bd4143cbec9a4eee, user_id=72a03b9c60724a88b42e4b27bdfe0dab, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:55.544508 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2695f70-e102-4946-be77-f6b3684ad440 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:55.613855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1384/2745] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:46:55 2022] PUT /identity/v3/projects/c59f48b88ad24399bd4143cbec9a4eee/users/72a03b9c60724a88b42e4b27bdfe0dab/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:46:55.623933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d92a45-ac56-44fe-848f-a27b5258c948 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.624227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d92a45-ac56-44fe-848f-a27b5258c948 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.624513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d92a45-ac56-44fe-848f-a27b5258c948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.651417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-60d92a45-ac56-44fe-848f-a27b5258c948 None None] MFA Rules not processed for user `72a03b9c60724a88b42e4b27bdfe0dab`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:46:55.652639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-60d92a45-ac56-44fe-848f-a27b5258c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:46:55.715117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1362/2746] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:46:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:46:55.719555 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695f70-e102-4946-be77-f6b3684ad440 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:55.733463 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:46:55.782340 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695f70-e102-4946-be77-f6b3684ad440 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:55.787022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4e1e-3afa-4e9f-885a-d4b3890d7054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:55.787672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4e1e-3afa-4e9f-885a-d4b3890d7054 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.787957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4e1e-3afa-4e9f-885a-d4b3890d7054 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.788235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4e1e-3afa-4e9f-885a-d4b3890d7054 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.788974 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d4e1e-3afa-4e9f-885a-d4b3890d7054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:55.791802 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d4e1e-3afa-4e9f-885a-d4b3890d7054 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:55.854465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1385/2747] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:46:55 2022] POST /identity/v3/projects => generated 401 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:46:55.858666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d92a45-ac56-44fe-848f-a27b5258c948 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:55.869120 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d92a45-ac56-44fe-848f-a27b5258c948 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:55.925144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507165c-264a-4cf2-836b-678603933b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:55.925818 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507165c-264a-4cf2-836b-678603933b26 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:55.926105 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507165c-264a-4cf2-836b-678603933b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:55.926384 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507165c-264a-4cf2-836b-678603933b26 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:55.927067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507165c-264a-4cf2-836b-678603933b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:55.928985 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507165c-264a-4cf2-836b-678603933b26 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:56.002021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1363/2748] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:46:55 2022] POST /identity/v3/users => generated 307 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:46:56.005993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4e1e-3afa-4e9f-885a-d4b3890d7054 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:56.017475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4e1e-3afa-4e9f-885a-d4b3890d7054 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:56.022233 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e10da-cb46-4366-ba7f-836a4b1fbcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:56.022898 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e10da-cb46-4366-ba7f-836a4b1fbcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:56.023188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e10da-cb46-4366-ba7f-836a4b1fbcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:56.023471 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e10da-cb46-4366-ba7f-836a4b1fbcd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:56.024210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e10da-cb46-4366-ba7f-836a4b1fbcd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:56.026503 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e10da-cb46-4366-ba7f-836a4b1fbcd7 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:56.036949 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1386/2749] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:46:56.041216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507165c-264a-4cf2-836b-678603933b26 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:56.047919 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:46:56.051788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507165c-264a-4cf2-836b-678603933b26 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:56.057135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bac295-30ef-4684-8018-cd89bf749134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:56.058457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bac295-30ef-4684-8018-cd89bf749134 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:56.058758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bac295-30ef-4684-8018-cd89bf749134 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:56.059037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bac295-30ef-4684-8018-cd89bf749134 admin admin] PATH_INFO: `/v3/projects/25fec50cf6a54bcd9439ae2f15a5d147/users/3d7953d9342b4dc0990666c5f82270e7/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:56.075293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bac295-30ef-4684-8018-cd89bf749134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25fec50cf6a54bcd9439ae2f15a5d147, user_id=3d7953d9342b4dc0990666c5f82270e7, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:56.077966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bac295-30ef-4684-8018-cd89bf749134 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:56.167954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1364/2750] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:46:56 2022] PUT /identity/v3/projects/25fec50cf6a54bcd9439ae2f15a5d147/users/3d7953d9342b4dc0990666c5f82270e7/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:46:56.172030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e10da-cb46-4366-ba7f-836a4b1fbcd7 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:56.182025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e10da-cb46-4366-ba7f-836a4b1fbcd7 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:56.239097 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eedde1-bc76-4046-af44-467c7d34ba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:56.239773 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eedde1-bc76-4046-af44-467c7d34ba5f admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:56.240057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eedde1-bc76-4046-af44-467c7d34ba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:56.240333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eedde1-bc76-4046-af44-467c7d34ba5f admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:56.241087 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eedde1-bc76-4046-af44-467c7d34ba5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:56.242705 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eedde1-bc76-4046-af44-467c7d34ba5f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:56.253102 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1387/2751] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:46:56.257211 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bac295-30ef-4684-8018-cd89bf749134 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:56.268025 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bac295-30ef-4684-8018-cd89bf749134 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:56.272880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:56.273747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:56.274034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:56.274321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] PATH_INFO: `/v3/projects/25fec50cf6a54bcd9439ae2f15a5d147/users/3d7953d9342b4dc0990666c5f82270e7/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:56.276285 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=25fec50cf6a54bcd9439ae2f15a5d147, user_id=3d7953d9342b4dc0990666c5f82270e7, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:56.278938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:56.289123 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1365/2752] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:46:56 2022] PUT /identity/v3/projects/25fec50cf6a54bcd9439ae2f15a5d147/users/3d7953d9342b4dc0990666c5f82270e7/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:46:56.300234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ad74f-d961-4e31-8a30-33df4b24e8cd None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:56.300541 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ad74f-d961-4e31-8a30-33df4b24e8cd None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:56.300828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ad74f-d961-4e31-8a30-33df4b24e8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:56.328215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-9b4ad74f-d961-4e31-8a30-33df4b24e8cd None None] MFA Rules not processed for user `3d7953d9342b4dc0990666c5f82270e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:46:56.329494 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-9b4ad74f-d961-4e31-8a30-33df4b24e8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:46:56.396084 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1388/2753] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:46:56 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:46:56.407358 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:56.409063 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:46:56.465856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c063602-a381-4b06-ad15-894457b564ee admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:56.469478 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f100-65ef-4d91-86df-9f5f1fa729bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:56.470122 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f100-65ef-4d91-86df-9f5f1fa729bd service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:56.470418 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f100-65ef-4d91-86df-9f5f1fa729bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:56.470894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f100-65ef-4d91-86df-9f5f1fa729bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:56.481298 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944f100-65ef-4d91-86df-9f5f1fa729bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:56.482732 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944f100-65ef-4d91-86df-9f5f1fa729bd service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:56.493386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1366/2754] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:46:56 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:46:56.782158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ad74f-d961-4e31-8a30-33df4b24e8cd None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:46:56.793202 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ad74f-d961-4e31-8a30-33df4b24e8cd None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:46:56.796858 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f0968-2e9b-49f5-bfec-87b2a2f0fa6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:46:56.797513 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0968-2e9b-49f5-bfec-87b2a2f0fa6b service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:46:56.797797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0968-2e9b-49f5-bfec-87b2a2f0fa6b service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:46:56.798074 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0968-2e9b-49f5-bfec-87b2a2f0fa6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:46:56.803519 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:46:56.855842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f0968-2e9b-49f5-bfec-87b2a2f0fa6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:46:56.857543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f0968-2e9b-49f5-bfec-87b2a2f0fa6b service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:46:56.867217 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1389/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:46:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:03.689741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f100-65ef-4d91-86df-9f5f1fa729bd service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:03.755217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f100-65ef-4d91-86df-9f5f1fa729bd service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:03.759887 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:03.761532 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:03.761703 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:03.761984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] PATH_INFO: `/v3/users/72a03b9c60724a88b42e4b27bdfe0dab` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:03.763182 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a03b9c60724a88b42e4b27bdfe0dab)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:03.765135 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:03.952363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a03b9c60724a88b42e4b27bdfe0dab'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:03.959146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a03b9c60724a88b42e4b27bdfe0dab'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:03.959741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a03b9c60724a88b42e4b27bdfe0dab'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:03.960377 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a03b9c60724a88b42e4b27bdfe0dab'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:04.047356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a03b9c60724a88b42e4b27bdfe0dab'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:04.054886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:04.066354 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1367/2756] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:47:03 2022] DELETE /identity/v3/users/72a03b9c60724a88b42e4b27bdfe0dab => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:04.070830 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522943-9698-4255-9904-84a1b3de49d8 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:04.071271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522943-9698-4255-9904-84a1b3de49d8 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:04.071484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522943-9698-4255-9904-84a1b3de49d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:04.089424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-5e522943-9698-4255-9904-84a1b3de49d8 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:04.090629 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5e522943-9698-4255-9904-84a1b3de49d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:04.149288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1390/2757] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:04.159295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:04.160742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:04.218631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a8993-8dbc-4279-bb58-f5b044d8c782 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:04.222163 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7743a0-905e-43f4-bd2b-dcf647b75ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:04.222923 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7743a0-905e-43f4-bd2b-dcf647b75ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:04.223111 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7743a0-905e-43f4-bd2b-dcf647b75ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:04.223391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7743a0-905e-43f4-bd2b-dcf647b75ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:04.232971 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7743a0-905e-43f4-bd2b-dcf647b75ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:04.234411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7743a0-905e-43f4-bd2b-dcf647b75ad4 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:04.248717 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1368/2758] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:04.648833 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e522943-9698-4255-9904-84a1b3de49d8 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:04.654215 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-5e522943-9698-4255-9904-84a1b3de49d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:04.663061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e522943-9698-4255-9904-84a1b3de49d8 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:04.668684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:04.669605 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:04.669896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:04.670179 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] PATH_INFO: `/v3/projects/c59f48b88ad24399bd4143cbec9a4eee` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:04.671445 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59f48b88ad24399bd4143cbec9a4eee)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:04.673405 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:04.779717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59f48b88ad24399bd4143cbec9a4eee', 'request_id': 'req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:04.829866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59f48b88ad24399bd4143cbec9a4eee', 'request_id': 'req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:04.835730 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59f48b88ad24399bd4143cbec9a4eee', 'request_id': 'req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:04.847666 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] The token cache is being invalidate because project c59f48b88ad24399bd4143cbec9a4eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:47:04.848001 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:04.848692 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:04.850226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1391/2759] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:47:04 2022] DELETE /identity/v3/projects/c59f48b88ad24399bd4143cbec9a4eee => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:47:04.854168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7743a0-905e-43f4-bd2b-dcf647b75ad4 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:04.863778 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7743a0-905e-43f4-bd2b-dcf647b75ad4 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:04.919101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:04.919969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:04.920257 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:04.920563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] PATH_INFO: `/v3/users/3d7953d9342b4dc0990666c5f82270e7` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:04.921751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7953d9342b4dc0990666c5f82270e7)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:04.923658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:05.101754 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7953d9342b4dc0990666c5f82270e7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.108579 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7953d9342b4dc0990666c5f82270e7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.109327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7953d9342b4dc0990666c5f82270e7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.109928 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7953d9342b4dc0990666c5f82270e7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.184648 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-27bf5dcc-d1d8-4980-9191-a3cb659f6648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7953d9342b4dc0990666c5f82270e7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.202915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1369/2760] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:47:04 2022] DELETE /identity/v3/users/3d7953d9342b4dc0990666c5f82270e7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:05.563896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:05.565427 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:05.578860 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07c285-1b56-410c-b909-a4c5d7c2f0a1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:05.609553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:05.637178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:05.638044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:05.638356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:05.638642 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] PATH_INFO: `/v3/projects/25fec50cf6a54bcd9439ae2f15a5d147` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:05.639850 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25fec50cf6a54bcd9439ae2f15a5d147)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:05.641826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:05.749571 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25fec50cf6a54bcd9439ae2f15a5d147', 'request_id': 'req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.818543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25fec50cf6a54bcd9439ae2f15a5d147', 'request_id': 'req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.824416 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25fec50cf6a54bcd9439ae2f15a5d147', 'request_id': 'req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.835680 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] The token cache is being invalidate because project 25fec50cf6a54bcd9439ae2f15a5d147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:47:05.835966 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.836622 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-735b4d2a-757e-4ea1-a6fe-3c1d6c73e807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:05.838146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1392/2761] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:47:05 2022] DELETE /identity/v3/projects/25fec50cf6a54bcd9439ae2f15a5d147 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:05.842963 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3586d-5abf-4c1b-8eb7-0a489cc30144 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:05.843260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3586d-5abf-4c1b-8eb7-0a489cc30144 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:05.843546 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3586d-5abf-4c1b-8eb7-0a489cc30144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:05.862356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-3bf3586d-5abf-4c1b-8eb7-0a489cc30144 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:05.863615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3bf3586d-5abf-4c1b-8eb7-0a489cc30144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:05.868896 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:05.923074 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1370/2762] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:05.930442 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe69c5-b283-4578-9a34-d3c1ea0a556a None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:05.930739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe69c5-b283-4578-9a34-d3c1ea0a556a None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:05.931049 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe69c5-b283-4578-9a34-d3c1ea0a556a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:05.949428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-61fe69c5-b283-4578-9a34-d3c1ea0a556a None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:05.950945 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-61fe69c5-b283-4578-9a34-d3c1ea0a556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:05.958739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1393/2763] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:05.962871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3586d-5abf-4c1b-8eb7-0a489cc30144 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:05.973060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3586d-5abf-4c1b-8eb7-0a489cc30144 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:05.977759 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:05.978406 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:05.978696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:05.978978 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:05.979705 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:47:05.980018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:05.982260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:05.993716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1371/2764] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:47:05.998008 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe69c5-b283-4578-9a34-d3c1ea0a556a None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.007312 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe69c5-b283-4578-9a34-d3c1ea0a556a None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.011900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041501-296a-4146-bea5-520ab08cb6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.012568 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041501-296a-4146-bea5-520ab08cb6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.012861 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041501-296a-4146-bea5-520ab08cb6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.013131 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041501-296a-4146-bea5-520ab08cb6f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.013825 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041501-296a-4146-bea5-520ab08cb6f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.015746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041501-296a-4146-bea5-520ab08cb6f8 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:06.094621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1394/2765] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:47:05 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:06.098681 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.109255 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550820ab-95da-46f3-a5b3-756ab02b8130 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.162868 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebd576-f75c-485b-a5af-a4e5e753ce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.163526 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebd576-f75c-485b-a5af-a4e5e753ce51 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.163909 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebd576-f75c-485b-a5af-a4e5e753ce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.164198 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebd576-f75c-485b-a5af-a4e5e753ce51 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.164913 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebd576-f75c-485b-a5af-a4e5e753ce51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.166792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebd576-f75c-485b-a5af-a4e5e753ce51 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:06.241342 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1372/2766] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:47:06 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:06.245339 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041501-296a-4146-bea5-520ab08cb6f8 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.256038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041501-296a-4146-bea5-520ab08cb6f8 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.260896 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf826fa-eb60-4a1e-9646-5e6e87df13d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.261540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf826fa-eb60-4a1e-9646-5e6e87df13d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.261824 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf826fa-eb60-4a1e-9646-5e6e87df13d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.262100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf826fa-eb60-4a1e-9646-5e6e87df13d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.262823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf826fa-eb60-4a1e-9646-5e6e87df13d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.264431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf826fa-eb60-4a1e-9646-5e6e87df13d6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:06.274739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1395/2767] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:06.282060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebd576-f75c-485b-a5af-a4e5e753ce51 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.291973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebd576-f75c-485b-a5af-a4e5e753ce51 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.297086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0809fe-4096-4d03-bbad-1e8753797f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.297775 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0809fe-4096-4d03-bbad-1e8753797f29 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.298053 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0809fe-4096-4d03-bbad-1e8753797f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.298330 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0809fe-4096-4d03-bbad-1e8753797f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.299048 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0809fe-4096-4d03-bbad-1e8753797f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.300670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0809fe-4096-4d03-bbad-1e8753797f29 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:06.310557 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1373/2768] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:06.314637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf826fa-eb60-4a1e-9646-5e6e87df13d6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.324396 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf826fa-eb60-4a1e-9646-5e6e87df13d6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.329770 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef413f-719e-41e1-b8b9-c31b35c76ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.330631 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef413f-719e-41e1-b8b9-c31b35c76ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.330917 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef413f-719e-41e1-b8b9-c31b35c76ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.331214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef413f-719e-41e1-b8b9-c31b35c76ef7 admin admin] PATH_INFO: `/v3/projects/29a7145000c14b01817e4ed8645ae1d6/users/ece99c9e2a894611814b2ace1e4a28f6/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.348257 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef413f-719e-41e1-b8b9-c31b35c76ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a7145000c14b01817e4ed8645ae1d6, user_id=ece99c9e2a894611814b2ace1e4a28f6, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.349923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef413f-719e-41e1-b8b9-c31b35c76ef7 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:06.430612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1396/2769] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:06 2022] PUT /identity/v3/projects/29a7145000c14b01817e4ed8645ae1d6/users/ece99c9e2a894611814b2ace1e4a28f6/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:06.440841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc806b1d-ab57-4c4e-8294-956af297d0c0 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.441136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc806b1d-ab57-4c4e-8294-956af297d0c0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.441447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc806b1d-ab57-4c4e-8294-956af297d0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.471586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-fc806b1d-ab57-4c4e-8294-956af297d0c0 None None] MFA Rules not processed for user `ece99c9e2a894611814b2ace1e4a28f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:06.472806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fc806b1d-ab57-4c4e-8294-956af297d0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:06.535213 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1374/2770] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:06.539656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef413f-719e-41e1-b8b9-c31b35c76ef7 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.598751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:06.602380 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef413f-719e-41e1-b8b9-c31b35c76ef7 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.607094 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85065c26-6279-4a95-b3d4-7cab8e666f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.609668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85065c26-6279-4a95-b3d4-7cab8e666f2f admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.609668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85065c26-6279-4a95-b3d4-7cab8e666f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.609668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85065c26-6279-4a95-b3d4-7cab8e666f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.609668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85065c26-6279-4a95-b3d4-7cab8e666f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.611088 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85065c26-6279-4a95-b3d4-7cab8e666f2f admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:06.701407 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1397/2771] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:47:06 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:06.705604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc806b1d-ab57-4c4e-8294-956af297d0c0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.722964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc806b1d-ab57-4c4e-8294-956af297d0c0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.728743 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:06.782616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e852e3e-fcb8-48ce-99c4-87a94744ba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.783304 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e852e3e-fcb8-48ce-99c4-87a94744ba8b admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.783595 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e852e3e-fcb8-48ce-99c4-87a94744ba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.783882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e852e3e-fcb8-48ce-99c4-87a94744ba8b admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.784623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e852e3e-fcb8-48ce-99c4-87a94744ba8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.789004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e852e3e-fcb8-48ce-99c4-87a94744ba8b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:06.893950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1375/2772] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:47:06 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:47:06.897659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85065c26-6279-4a95-b3d4-7cab8e666f2f admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.907929 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85065c26-6279-4a95-b3d4-7cab8e666f2f admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.912435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5032b-1263-4f2a-b415-b4ec3403f1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.913264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5032b-1263-4f2a-b415-b4ec3403f1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.913558 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5032b-1263-4f2a-b415-b4ec3403f1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.913828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5032b-1263-4f2a-b415-b4ec3403f1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.914547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5032b-1263-4f2a-b415-b4ec3403f1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.916667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5032b-1263-4f2a-b415-b4ec3403f1e6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:06.927495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1398/2773] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:06.931623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e852e3e-fcb8-48ce-99c4-87a94744ba8b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:06.942108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e852e3e-fcb8-48ce-99c4-87a94744ba8b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:06.946893 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960bb12-0475-444e-ab42-3b86fed5d537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:06.951088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960bb12-0475-444e-ab42-3b86fed5d537 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:06.951088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960bb12-0475-444e-ab42-3b86fed5d537 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:06.951088 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960bb12-0475-444e-ab42-3b86fed5d537 admin admin] PATH_INFO: `/v3/projects/3de7517111664d2bb5a26720ab32579c/users/81fffd6552a04e3ba2b382b81f659de8/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:06.964084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960bb12-0475-444e-ab42-3b86fed5d537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de7517111664d2bb5a26720ab32579c, user_id=81fffd6552a04e3ba2b382b81f659de8, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:06.967328 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960bb12-0475-444e-ab42-3b86fed5d537 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:07.052845 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1376/2774] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:06 2022] PUT /identity/v3/projects/3de7517111664d2bb5a26720ab32579c/users/81fffd6552a04e3ba2b382b81f659de8/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:07.056743 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5032b-1263-4f2a-b415-b4ec3403f1e6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:07.066952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5032b-1263-4f2a-b415-b4ec3403f1e6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:07.122818 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcdd5b-7433-4aee-86a6-fe6e08837b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:07.123510 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcdd5b-7433-4aee-86a6-fe6e08837b4b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:07.123794 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcdd5b-7433-4aee-86a6-fe6e08837b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:07.124065 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcdd5b-7433-4aee-86a6-fe6e08837b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:07.124979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcdd5b-7433-4aee-86a6-fe6e08837b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:07.126637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcdd5b-7433-4aee-86a6-fe6e08837b4b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:07.136145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1399/2775] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:07.140859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960bb12-0475-444e-ab42-3b86fed5d537 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:07.150929 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960bb12-0475-444e-ab42-3b86fed5d537 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:07.156035 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:07.156936 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:07.157216 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:07.157491 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] PATH_INFO: `/v3/projects/3de7517111664d2bb5a26720ab32579c/users/81fffd6552a04e3ba2b382b81f659de8/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:07.159465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de7517111664d2bb5a26720ab32579c, user_id=81fffd6552a04e3ba2b382b81f659de8, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:07.162129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:07.172472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1377/2776] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:07 2022] PUT /identity/v3/projects/3de7517111664d2bb5a26720ab32579c/users/81fffd6552a04e3ba2b382b81f659de8/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:47:07.189981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d26246-1cb0-4601-acbd-2387b9a3c0d0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:07.190273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d26246-1cb0-4601-acbd-2387b9a3c0d0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:07.190582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d26246-1cb0-4601-acbd-2387b9a3c0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:07.217435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-70d26246-1cb0-4601-acbd-2387b9a3c0d0 None None] MFA Rules not processed for user `81fffd6552a04e3ba2b382b81f659de8`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:07.218621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-70d26246-1cb0-4601-acbd-2387b9a3c0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:07.284843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1400/2777] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:07.304292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:07.305937 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:07.362363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fad5ba-083c-46d9-9cfa-847e9f752669 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:07.365981 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5049ad-f840-4f41-8a23-179c537e8988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:07.366610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5049ad-f840-4f41-8a23-179c537e8988 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:07.366895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5049ad-f840-4f41-8a23-179c537e8988 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:07.367175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5049ad-f840-4f41-8a23-179c537e8988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:07.376781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5049ad-f840-4f41-8a23-179c537e8988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:07.378265 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5049ad-f840-4f41-8a23-179c537e8988 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:07.387727 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1378/2778] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:07.486712 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d26246-1cb0-4601-acbd-2387b9a3c0d0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:07.497101 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d26246-1cb0-4601-acbd-2387b9a3c0d0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:07.499789 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b202a-d30d-4d0c-8a86-7d68b7f236eb None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:07.500094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b202a-d30d-4d0c-8a86-7d68b7f236eb None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:07.500454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b202a-d30d-4d0c-8a86-7d68b7f236eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:07.501254 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:07.501916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:07.502205 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:07.502487 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:07.523336 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-971b202a-d30d-4d0c-8a86-7d68b7f236eb None None] MFA Rules not processed for user `517928d85b764c089c6dd5b2d088639c`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:07.524618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-971b202a-d30d-4d0c-8a86-7d68b7f236eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:07.555279 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:07.564561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:07.566082 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:07.576197 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1401/2779] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:07.588679 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1379/2780] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:08.100294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:08.101767 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:08.168311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760d73-1339-4f39-ac6c-5317bb89e8a6 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:08.172286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:08.172973 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:08.173264 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:08.173564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:08.175483 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:08.186161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:08.187588 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:08.195745 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1402/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:08.814610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b202a-d30d-4d0c-8a86-7d68b7f236eb None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:08.816056 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-971b202a-d30d-4d0c-8a86-7d68b7f236eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:08.824302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:08.937142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b202a-d30d-4d0c-8a86-7d68b7f236eb None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:08.942223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118ecd9-cde4-4c39-b86f-5dda0ce46e05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:08.942931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118ecd9-cde4-4c39-b86f-5dda0ce46e05 service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:08.943219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118ecd9-cde4-4c39-b86f-5dda0ce46e05 service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:08.943502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118ecd9-cde4-4c39-b86f-5dda0ce46e05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:08.945311 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9118ecd9-cde4-4c39-b86f-5dda0ce46e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:08.956806 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118ecd9-cde4-4c39-b86f-5dda0ce46e05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:08.958509 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118ecd9-cde4-4c39-b86f-5dda0ce46e05 service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:08.968903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1380/2782] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:10.557210 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:10.651039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca81df-257e-442b-8f27-18de7a39b939 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:10.656251 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:10.657225 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:10.657535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:10.657816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] PATH_INFO: `/v3/users/ece99c9e2a894611814b2ace1e4a28f6` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:10.659076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece99c9e2a894611814b2ace1e4a28f6)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:10.661042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:10.841536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece99c9e2a894611814b2ace1e4a28f6'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:10.853739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece99c9e2a894611814b2ace1e4a28f6'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:10.860988 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece99c9e2a894611814b2ace1e4a28f6'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:10.861718 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece99c9e2a894611814b2ace1e4a28f6'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:10.862456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece99c9e2a894611814b2ace1e4a28f6'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:10.864234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:10.934817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1403/2783] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:47:10 2022] DELETE /identity/v3/users/ece99c9e2a894611814b2ace1e4a28f6 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:10.939276 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a3a3-4b4d-40bc-87df-7e075b335582 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:10.939788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a3a3-4b4d-40bc-87df-7e075b335582 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:10.939897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a3a3-4b4d-40bc-87df-7e075b335582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:10.986433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-df88a3a3-4b4d-40bc-87df-7e075b335582 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:10.987724 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-df88a3a3-4b4d-40bc-87df-7e075b335582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:11.076601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1381/2784] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:11.085477 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:11.087169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:11.169438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c2c79-0289-4504-9d78-f6446658efaf admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:11.173059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf040f-9166-425e-bbdf-bac6b9134ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:11.173709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf040f-9166-425e-bbdf-bac6b9134ece service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:11.173992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf040f-9166-425e-bbdf-bac6b9134ece service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:11.174269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf040f-9166-425e-bbdf-bac6b9134ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:11.184476 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf040f-9166-425e-bbdf-bac6b9134ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:11.186012 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf040f-9166-425e-bbdf-bac6b9134ece service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:11.196153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1404/2785] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:47:11.593915 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a3a3-4b4d-40bc-87df-7e075b335582 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:11.595551 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-df88a3a3-4b4d-40bc-87df-7e075b335582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:11.609945 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a3a3-4b4d-40bc-87df-7e075b335582 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:11.615888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:11.616803 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:11.617353 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:11.617637 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] PATH_INFO: `/v3/projects/29a7145000c14b01817e4ed8645ae1d6` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:11.618982 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a7145000c14b01817e4ed8645ae1d6)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:11.620989 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:11.729857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:11.747734 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a7145000c14b01817e4ed8645ae1d6', 'request_id': 'req-6cd6eb30-140e-4dc1-b35d-f5e034fab880'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:11.813174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a7145000c14b01817e4ed8645ae1d6', 'request_id': 'req-6cd6eb30-140e-4dc1-b35d-f5e034fab880'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:11.821151 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a7145000c14b01817e4ed8645ae1d6', 'request_id': 'req-6cd6eb30-140e-4dc1-b35d-f5e034fab880'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:11.838702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] The token cache is being invalidate because project 29a7145000c14b01817e4ed8645ae1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:47:11.838979 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:11.839686 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:11.841464 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1382/2786] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:47:11 2022] DELETE /identity/v3/projects/29a7145000c14b01817e4ed8645ae1d6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:11.845495 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf040f-9166-425e-bbdf-bac6b9134ece service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:11.862169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf040f-9166-425e-bbdf-bac6b9134ece service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:11.923107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:11.923990 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:11.924273 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:11.924573 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] PATH_INFO: `/v3/users/81fffd6552a04e3ba2b382b81f659de8` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:11.925795 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81fffd6552a04e3ba2b382b81f659de8)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:11.927728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:12.133570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81fffd6552a04e3ba2b382b81f659de8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:12.149553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81fffd6552a04e3ba2b382b81f659de8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:12.171715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81fffd6552a04e3ba2b382b81f659de8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:12.174709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81fffd6552a04e3ba2b382b81f659de8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:12.175551 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1853dda1-5487-4b4e-82af-a00a30740a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81fffd6552a04e3ba2b382b81f659de8'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:12.179592 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:12.286720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1405/2787] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:47:11 2022] DELETE /identity/v3/users/81fffd6552a04e3ba2b382b81f659de8 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:12.742307 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:12.743891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:12.756657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6eb30-140e-4dc1-b35d-f5e034fab880 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:12.828448 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:12.829337 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:12.830530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:12.830530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] PATH_INFO: `/v3/projects/3de7517111664d2bb5a26720ab32579c` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:12.831097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de7517111664d2bb5a26720ab32579c)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:12.833043 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:12.929005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de7517111664d2bb5a26720ab32579c', 'request_id': 'req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:12.995841 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de7517111664d2bb5a26720ab32579c', 'request_id': 'req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:13.002039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de7517111664d2bb5a26720ab32579c', 'request_id': 'req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:13.013492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] The token cache is being invalidate because project 3de7517111664d2bb5a26720ab32579c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:47:13.013788 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:13.014428 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c2ec1b2b-9def-4dc6-a5c4-43cf2224afa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:13.015995 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1383/2788] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:47:12 2022] DELETE /identity/v3/projects/3de7517111664d2bb5a26720ab32579c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:13.025119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466da7e-278c-4d85-b7b8-a4ada6fc5d26 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.025119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466da7e-278c-4d85-b7b8-a4ada6fc5d26 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.025119 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466da7e-278c-4d85-b7b8-a4ada6fc5d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.060289 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-2466da7e-278c-4d85-b7b8-a4ada6fc5d26 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:13.061575 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-2466da7e-278c-4d85-b7b8-a4ada6fc5d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:13.123382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1406/2789] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:47:13.131096 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0d85e-2572-4ce4-89a8-a32361be7d28 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.131387 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0d85e-2572-4ce4-89a8-a32361be7d28 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.131695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0d85e-2572-4ce4-89a8-a32361be7d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.150370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-8ef0d85e-2572-4ce4-89a8-a32361be7d28 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:13.151569 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-8ef0d85e-2572-4ce4-89a8-a32361be7d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:13.160097 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1384/2790] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:13.164319 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466da7e-278c-4d85-b7b8-a4ada6fc5d26 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:13.174409 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466da7e-278c-4d85-b7b8-a4ada6fc5d26 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:13.179499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:13.180178 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.180465 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.180807 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.181540 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:47:13.181851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:13.183484 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:13.195679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1407/2791] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:47:13.200921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0d85e-2572-4ce4-89a8-a32361be7d28 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:13.210174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0d85e-2572-4ce4-89a8-a32361be7d28 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:13.215266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6812b99-df95-4a73-9c42-da395917590b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:13.216501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6812b99-df95-4a73-9c42-da395917590b admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.216816 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6812b99-df95-4a73-9c42-da395917590b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.217129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6812b99-df95-4a73-9c42-da395917590b admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.217854 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6812b99-df95-4a73-9c42-da395917590b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:13.219751 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6812b99-df95-4a73-9c42-da395917590b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:13.327720 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1385/2792] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:47:13 2022] POST /identity/v3/projects => generated 407 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:13.329255 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:13.340918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533f56d-46ad-4be9-8318-2c2af3f6d8f5 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:13.405045 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e881b71-a557-4584-8d58-585352a48ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:13.405753 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881b71-a557-4584-8d58-585352a48ebc admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.406017 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881b71-a557-4584-8d58-585352a48ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.406294 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881b71-a557-4584-8d58-585352a48ebc admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.406992 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e881b71-a557-4584-8d58-585352a48ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:13.408993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e881b71-a557-4584-8d58-585352a48ebc admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:13.481639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1408/2793] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:47:13 2022] POST /identity/v3/users => generated 304 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:13.485700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6812b99-df95-4a73-9c42-da395917590b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:13.497777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6812b99-df95-4a73-9c42-da395917590b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:13.502380 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40bb03-3709-4f9e-86e5-c1c3008859c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:13.503148 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40bb03-3709-4f9e-86e5-c1c3008859c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.503433 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40bb03-3709-4f9e-86e5-c1c3008859c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.503737 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40bb03-3709-4f9e-86e5-c1c3008859c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.504465 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40bb03-3709-4f9e-86e5-c1c3008859c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:13.506287 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40bb03-3709-4f9e-86e5-c1c3008859c2 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:13.517351 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1386/2794] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:13.521526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e881b71-a557-4584-8d58-585352a48ebc admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:13.531979 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e881b71-a557-4584-8d58-585352a48ebc admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:13.536724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a0041-244d-4db1-9ef7-12a045e118ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:13.537327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a0041-244d-4db1-9ef7-12a045e118ea admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.537616 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a0041-244d-4db1-9ef7-12a045e118ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.537897 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a0041-244d-4db1-9ef7-12a045e118ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.538628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a0041-244d-4db1-9ef7-12a045e118ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:13.540235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a0041-244d-4db1-9ef7-12a045e118ea admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:13.550115 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1409/2795] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:13.554223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40bb03-3709-4f9e-86e5-c1c3008859c2 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:13.561061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:13.564741 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40bb03-3709-4f9e-86e5-c1c3008859c2 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:13.569352 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eefcc-2246-4d38-ba60-0665c0be6d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:13.570318 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eefcc-2246-4d38-ba60-0665c0be6d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.570485 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eefcc-2246-4d38-ba60-0665c0be6d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.570763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eefcc-2246-4d38-ba60-0665c0be6d66 admin admin] PATH_INFO: `/v3/projects/c92ca4af59b64fa79d78ca781d9ad4cf/users/01fff5d56e3f446e89859ec0208695b5/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.586108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7eefcc-2246-4d38-ba60-0665c0be6d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92ca4af59b64fa79d78ca781d9ad4cf, user_id=01fff5d56e3f446e89859ec0208695b5, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:13.588791 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7eefcc-2246-4d38-ba60-0665c0be6d66 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:13.652658 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1387/2796] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:13 2022] PUT /identity/v3/projects/c92ca4af59b64fa79d78ca781d9ad4cf/users/01fff5d56e3f446e89859ec0208695b5/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:13.663175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f3ca-cc02-46d2-8e45-100f1cc21650 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.663466 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f3ca-cc02-46d2-8e45-100f1cc21650 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.663750 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f3ca-cc02-46d2-8e45-100f1cc21650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.690527 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-8ed7f3ca-cc02-46d2-8e45-100f1cc21650 None None] MFA Rules not processed for user `01fff5d56e3f446e89859ec0208695b5`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:13.691731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-8ed7f3ca-cc02-46d2-8e45-100f1cc21650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:13.724886 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:13.757022 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1410/2797] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:13.761514 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eefcc-2246-4d38-ba60-0665c0be6d66 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:13.832343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eefcc-2246-4d38-ba60-0665c0be6d66 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:13.836977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19660acd-2257-46f4-bc69-eb8e566b734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:13.837640 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19660acd-2257-46f4-bc69-eb8e566b734c admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:13.837927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19660acd-2257-46f4-bc69-eb8e566b734c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:13.838208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19660acd-2257-46f4-bc69-eb8e566b734c admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:13.838903 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19660acd-2257-46f4-bc69-eb8e566b734c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:13.840866 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19660acd-2257-46f4-bc69-eb8e566b734c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:13.962849 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1388/2798] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:47:13 2022] POST /identity/v3/projects => generated 409 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:47:13.967020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7f3ca-cc02-46d2-8e45-100f1cc21650 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:13.977535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7f3ca-cc02-46d2-8e45-100f1cc21650 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:14.033787 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2da0-5f5e-4c15-ac9e-588a3fa60cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:14.034491 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2da0-5f5e-4c15-ac9e-588a3fa60cac admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:14.034778 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2da0-5f5e-4c15-ac9e-588a3fa60cac admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:14.035059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2da0-5f5e-4c15-ac9e-588a3fa60cac admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:14.035752 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd2da0-5f5e-4c15-ac9e-588a3fa60cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:14.037704 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd2da0-5f5e-4c15-ac9e-588a3fa60cac admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:14.167446 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1411/2799] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:47:13 2022] POST /identity/v3/users => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:14.171356 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19660acd-2257-46f4-bc69-eb8e566b734c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:14.181736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19660acd-2257-46f4-bc69-eb8e566b734c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:14.186895 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40aa87-00dd-477d-bf0f-3ebeee06665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:14.187543 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40aa87-00dd-477d-bf0f-3ebeee06665b admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:14.187829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40aa87-00dd-477d-bf0f-3ebeee06665b admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:14.188108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40aa87-00dd-477d-bf0f-3ebeee06665b admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:14.188867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40aa87-00dd-477d-bf0f-3ebeee06665b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:14.190434 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40aa87-00dd-477d-bf0f-3ebeee06665b admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:14.201652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1389/2800] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:14.205598 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2da0-5f5e-4c15-ac9e-588a3fa60cac admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:14.215675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2da0-5f5e-4c15-ac9e-588a3fa60cac admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:14.220241 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161e297-421f-47ed-aaf0-27f85694aea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:14.221163 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161e297-421f-47ed-aaf0-27f85694aea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:14.221444 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161e297-421f-47ed-aaf0-27f85694aea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:14.221719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161e297-421f-47ed-aaf0-27f85694aea1 admin admin] PATH_INFO: `/v3/projects/d6aa232bfc5a414eae61219bee6b44d1/users/7286c63929474130875db3e8479afbbd/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:14.236067 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161e297-421f-47ed-aaf0-27f85694aea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6aa232bfc5a414eae61219bee6b44d1, user_id=7286c63929474130875db3e8479afbbd, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:14.238799 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161e297-421f-47ed-aaf0-27f85694aea1 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:14.309915 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1412/2801] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:14 2022] PUT /identity/v3/projects/d6aa232bfc5a414eae61219bee6b44d1/users/7286c63929474130875db3e8479afbbd/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:14.313810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40aa87-00dd-477d-bf0f-3ebeee06665b admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:14.323572 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40aa87-00dd-477d-bf0f-3ebeee06665b admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:14.383005 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84122f2a-f7b5-4ead-b36a-a34e02672bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:14.383689 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84122f2a-f7b5-4ead-b36a-a34e02672bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:14.383973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84122f2a-f7b5-4ead-b36a-a34e02672bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:14.384252 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84122f2a-f7b5-4ead-b36a-a34e02672bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:14.385021 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84122f2a-f7b5-4ead-b36a-a34e02672bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:14.386690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84122f2a-f7b5-4ead-b36a-a34e02672bb6 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:14.398347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1390/2802] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:14.402385 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161e297-421f-47ed-aaf0-27f85694aea1 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:14.412781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161e297-421f-47ed-aaf0-27f85694aea1 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:14.417368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:14.418313 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:14.418535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:14.418817 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] PATH_INFO: `/v3/projects/d6aa232bfc5a414eae61219bee6b44d1/users/7286c63929474130875db3e8479afbbd/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:14.420783 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6aa232bfc5a414eae61219bee6b44d1, user_id=7286c63929474130875db3e8479afbbd, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:14.423406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:14.433542 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1413/2803] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:14 2022] PUT /identity/v3/projects/d6aa232bfc5a414eae61219bee6b44d1/users/7286c63929474130875db3e8479afbbd/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:14.445186 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571db5ed-eefc-42f4-86ad-b198ec6327e1 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:14.445501 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571db5ed-eefc-42f4-86ad-b198ec6327e1 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:14.445786 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571db5ed-eefc-42f4-86ad-b198ec6327e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:14.472941 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-571db5ed-eefc-42f4-86ad-b198ec6327e1 None None] MFA Rules not processed for user `7286c63929474130875db3e8479afbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:14.474137 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-571db5ed-eefc-42f4-86ad-b198ec6327e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:14.516355 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:14.544571 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1391/2804] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:14.555286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:14.557153 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:14.620184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d93f8-a7c0-45a6-b153-d0c286553a7d admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:14.624683 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:14.625348 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:14.625638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:14.625918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:14.635464 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:14.636940 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:14.646368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1414/2805] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:14.963254 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571db5ed-eefc-42f4-86ad-b198ec6327e1 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:14.973480 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571db5ed-eefc-42f4-86ad-b198ec6327e1 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:14.977028 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f47e54-7cdf-45b9-a422-6c54eea6e3a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:14.977667 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f47e54-7cdf-45b9-a422-6c54eea6e3a5 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:14.977984 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f47e54-7cdf-45b9-a422-6c54eea6e3a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:14.978266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f47e54-7cdf-45b9-a422-6c54eea6e3a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:15.034346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f47e54-7cdf-45b9-a422-6c54eea6e3a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:15.035776 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f47e54-7cdf-45b9-a422-6c54eea6e3a5 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:15.045201 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1392/2806] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:14 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:23.799655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:23.801168 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:23.833866 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:23.875373 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600ba79-34a1-4b8e-bd86-20b433d83e24 service cinder] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:23.881044 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:23.881687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:23.881970 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:23.882250 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:23.884005 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:23.941170 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:23.942766 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:23.951856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1415/2807] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:24.462116 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f47e54-7cdf-45b9-a422-6c54eea6e3a5 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:24.525386 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f47e54-7cdf-45b9-a422-6c54eea6e3a5 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:24.530175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:24.531044 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:24.532052 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:24.532195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] PATH_INFO: `/v3/users/01fff5d56e3f446e89859ec0208695b5` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:24.533461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01fff5d56e3f446e89859ec0208695b5)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:24.536170 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:24.667043 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:24.668567 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:24.670700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01fff5d56e3f446e89859ec0208695b5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:24.678249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01fff5d56e3f446e89859ec0208695b5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:24.679259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01fff5d56e3f446e89859ec0208695b5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:24.679880 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01fff5d56e3f446e89859ec0208695b5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:24.683081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4f566-7165-4050-908b-f3fa4aa4773b service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:24.689684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72339190-4dbf-469e-aef7-6ae13d27521b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:24.690329 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339190-4dbf-469e-aef7-6ae13d27521b service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:24.690612 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339190-4dbf-469e-aef7-6ae13d27521b service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:24.690887 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339190-4dbf-469e-aef7-6ae13d27521b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:24.692720 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-72339190-4dbf-469e-aef7-6ae13d27521b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:24.707500 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72339190-4dbf-469e-aef7-6ae13d27521b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:24.709328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72339190-4dbf-469e-aef7-6ae13d27521b service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:24.720358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1416/2808] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:24.727702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01fff5d56e3f446e89859ec0208695b5'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:24.739648 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:24.748084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1393/2809] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:47:24 2022] DELETE /identity/v3/users/01fff5d56e3f446e89859ec0208695b5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:24.754019 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5adaa-fc97-4329-a2d6-4d723a13d370 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:24.754322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5adaa-fc97-4329-a2d6-4d723a13d370 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:24.754668 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5adaa-fc97-4329-a2d6-4d723a13d370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:24.773233 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-61a5adaa-fc97-4329-a2d6-4d723a13d370 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:24.774440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-61a5adaa-fc97-4329-a2d6-4d723a13d370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:24.836976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1417/2810] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:24.846692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:24.848119 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:24.906361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5211-9933-4ca7-96d3-6d06424b903c admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:24.910108 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f4fb3-8c76-4db7-849e-d79423e20a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:24.910753 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f4fb3-8c76-4db7-849e-d79423e20a66 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:24.911041 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f4fb3-8c76-4db7-849e-d79423e20a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:24.911322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f4fb3-8c76-4db7-849e-d79423e20a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:24.920484 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f4fb3-8c76-4db7-849e-d79423e20a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:24.921892 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f4fb3-8c76-4db7-849e-d79423e20a66 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:24.931487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1394/2811] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:25.402877 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5adaa-fc97-4329-a2d6-4d723a13d370 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:25.404325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-61a5adaa-fc97-4329-a2d6-4d723a13d370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:25.423526 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5adaa-fc97-4329-a2d6-4d723a13d370 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:25.428227 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:25.429276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:25.429435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:25.429713 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] PATH_INFO: `/v3/projects/c92ca4af59b64fa79d78ca781d9ad4cf` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:25.431038 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c92ca4af59b64fa79d78ca781d9ad4cf)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:25.433130 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:25.560759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:25.562158 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c92ca4af59b64fa79d78ca781d9ad4cf', 'request_id': 'req-1da6ae7a-1f96-4064-9368-db2f6e3d5565'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:25.692637 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c92ca4af59b64fa79d78ca781d9ad4cf', 'request_id': 'req-1da6ae7a-1f96-4064-9368-db2f6e3d5565'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:25.698418 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c92ca4af59b64fa79d78ca781d9ad4cf', 'request_id': 'req-1da6ae7a-1f96-4064-9368-db2f6e3d5565'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:25.710336 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] The token cache is being invalidate because project c92ca4af59b64fa79d78ca781d9ad4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:47:25.710864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:25.711826 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:25.713433 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1418/2812] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:47:25 2022] DELETE /identity/v3/projects/c92ca4af59b64fa79d78ca781d9ad4cf => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:25.717457 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f4fb3-8c76-4db7-849e-d79423e20a66 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:25.727495 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f4fb3-8c76-4db7-849e-d79423e20a66 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:25.782619 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:25.783481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:25.783766 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:25.784045 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] PATH_INFO: `/v3/users/7286c63929474130875db3e8479afbbd` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:25.785247 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7286c63929474130875db3e8479afbbd)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:25.787161 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:25.997852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7286c63929474130875db3e8479afbbd'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.005376 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7286c63929474130875db3e8479afbbd'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.006090 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7286c63929474130875db3e8479afbbd'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.006700 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7286c63929474130875db3e8479afbbd'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.076024 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-f39fc40e-c4b7-4740-92d1-475ffbdaae1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7286c63929474130875db3e8479afbbd'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.093392 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1395/2813] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:47:25 2022] DELETE /identity/v3/users/7286c63929474130875db3e8479afbbd => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:26.500359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:26.502079 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:26.514840 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6ae7a-1f96-4064-9368-db2f6e3d5565 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:26.570727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:26.571632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:26.571918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:26.572194 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] PATH_INFO: `/v3/projects/d6aa232bfc5a414eae61219bee6b44d1` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:26.573404 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6aa232bfc5a414eae61219bee6b44d1)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:26.575301 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:26.699553 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6aa232bfc5a414eae61219bee6b44d1', 'request_id': 'req-d260cd2c-2f82-447a-8327-41ae98a1e089'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.778871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6aa232bfc5a414eae61219bee6b44d1', 'request_id': 'req-d260cd2c-2f82-447a-8327-41ae98a1e089'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.785359 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6aa232bfc5a414eae61219bee6b44d1', 'request_id': 'req-d260cd2c-2f82-447a-8327-41ae98a1e089'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.796828 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] The token cache is being invalidate because project d6aa232bfc5a414eae61219bee6b44d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:47:26.797118 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.797797 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-d260cd2c-2f82-447a-8327-41ae98a1e089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:26.799333 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1419/2814] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:47:26 2022] DELETE /identity/v3/projects/d6aa232bfc5a414eae61219bee6b44d1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:26.804192 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72f382-0d06-45c3-8bfc-51cbbe68f175 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:26.804691 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72f382-0d06-45c3-8bfc-51cbbe68f175 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:26.804792 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72f382-0d06-45c3-8bfc-51cbbe68f175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:26.828709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:26.834671 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-1d72f382-0d06-45c3-8bfc-51cbbe68f175 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:26.835897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-1d72f382-0d06-45c3-8bfc-51cbbe68f175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:26.894825 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1396/2815] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:26.902599 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014a61a-f58f-4c8c-9029-80d9c8294258 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:26.902893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014a61a-f58f-4c8c-9029-80d9c8294258 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:26.903175 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014a61a-f58f-4c8c-9029-80d9c8294258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:26.921000 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-f014a61a-f58f-4c8c-9029-80d9c8294258 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:26.922200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-f014a61a-f58f-4c8c-9029-80d9c8294258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:26.930033 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1420/2816] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:26.934065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72f382-0d06-45c3-8bfc-51cbbe68f175 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:26.944081 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72f382-0d06-45c3-8bfc-51cbbe68f175 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:26.948722 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:26.949368 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:26.949649 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:26.949924 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:26.950922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:47:26.951227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:26.952833 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:26.964586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1397/2817] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:47:26.968585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014a61a-f58f-4c8c-9029-80d9c8294258 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:26.977639 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014a61a-f58f-4c8c-9029-80d9c8294258 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:26.982220 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62079346-e890-4206-86be-a8108a3ec617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:26.982991 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62079346-e890-4206-86be-a8108a3ec617 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:26.983276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62079346-e890-4206-86be-a8108a3ec617 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:26.983556 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62079346-e890-4206-86be-a8108a3ec617 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:26.984238 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62079346-e890-4206-86be-a8108a3ec617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:26.986159 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62079346-e890-4206-86be-a8108a3ec617 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:27.059723 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1421/2818] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:47:26 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:27.064321 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:27.074481 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7c218-a5ce-4541-bcbf-b33ca5473c9f admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:27.132288 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5eb0c-5015-46db-b6a4-145afd078d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:27.133166 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5eb0c-5015-46db-b6a4-145afd078d58 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:27.133564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5eb0c-5015-46db-b6a4-145afd078d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:27.133938 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5eb0c-5015-46db-b6a4-145afd078d58 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:27.134790 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5eb0c-5015-46db-b6a4-145afd078d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:27.137062 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5eb0c-5015-46db-b6a4-145afd078d58 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:27.213731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1398/2819] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:47:27 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:27.216428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62079346-e890-4206-86be-a8108a3ec617 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:27.226746 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62079346-e890-4206-86be-a8108a3ec617 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:27.231288 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18a2a-38d6-4e35-9ca6-a3a5b8012774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:27.231947 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18a2a-38d6-4e35-9ca6-a3a5b8012774 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:27.232235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18a2a-38d6-4e35-9ca6-a3a5b8012774 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:27.232534 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18a2a-38d6-4e35-9ca6-a3a5b8012774 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:27.236810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b18a2a-38d6-4e35-9ca6-a3a5b8012774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:27.236810 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b18a2a-38d6-4e35-9ca6-a3a5b8012774 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:27.245586 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1422/2820] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:27.249391 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5eb0c-5015-46db-b6a4-145afd078d58 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:27.260083 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5eb0c-5015-46db-b6a4-145afd078d58 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:27.264652 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b351f3-dcaf-4261-ac44-be1a3fe3bbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:27.265296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b351f3-dcaf-4261-ac44-be1a3fe3bbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:27.265582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b351f3-dcaf-4261-ac44-be1a3fe3bbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:27.265860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b351f3-dcaf-4261-ac44-be1a3fe3bbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:27.266624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b351f3-dcaf-4261-ac44-be1a3fe3bbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:27.268205 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b351f3-dcaf-4261-ac44-be1a3fe3bbd8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:27.277891 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1399/2821] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:27.281977 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18a2a-38d6-4e35-9ca6-a3a5b8012774 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:27.291714 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18a2a-38d6-4e35-9ca6-a3a5b8012774 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:27.296248 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5850ad8-8ddd-4ff8-bc8c-5571345c21ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:27.297144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5850ad8-8ddd-4ff8-bc8c-5571345c21ef admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:27.297428 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5850ad8-8ddd-4ff8-bc8c-5571345c21ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:27.298200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5850ad8-8ddd-4ff8-bc8c-5571345c21ef admin admin] PATH_INFO: `/v3/projects/3003d2da94f148629701a517a70f35ec/users/ad2e5711551e49c3862a2c7d76e98f71/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:27.379285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5850ad8-8ddd-4ff8-bc8c-5571345c21ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3003d2da94f148629701a517a70f35ec, user_id=ad2e5711551e49c3862a2c7d76e98f71, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:27.379285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5850ad8-8ddd-4ff8-bc8c-5571345c21ef admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:27.493831 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1423/2822] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:27 2022] PUT /identity/v3/projects/3003d2da94f148629701a517a70f35ec/users/ad2e5711551e49c3862a2c7d76e98f71/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:27.505742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baad3c2-7ec7-4a39-b747-833780a918e0 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:27.506034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baad3c2-7ec7-4a39-b747-833780a918e0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:27.506322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baad3c2-7ec7-4a39-b747-833780a918e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:27.533370 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6baad3c2-7ec7-4a39-b747-833780a918e0 None None] MFA Rules not processed for user `ad2e5711551e49c3862a2c7d76e98f71`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:27.534604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6baad3c2-7ec7-4a39-b747-833780a918e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:27.607502 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1400/2823] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:27.612675 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5850ad8-8ddd-4ff8-bc8c-5571345c21ef admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:27.617247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:27.676582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5850ad8-8ddd-4ff8-bc8c-5571345c21ef admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:27.682075 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b440c1-497d-450e-8827-dbf7fceae5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:27.682741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b440c1-497d-450e-8827-dbf7fceae5dd admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:27.683069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b440c1-497d-450e-8827-dbf7fceae5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:27.683356 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b440c1-497d-450e-8827-dbf7fceae5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:27.684039 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b440c1-497d-450e-8827-dbf7fceae5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:27.686013 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b440c1-497d-450e-8827-dbf7fceae5dd admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:27.817270 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1424/2824] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:47:27 2022] POST /identity/v3/projects => generated 381 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:27.818582 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baad3c2-7ec7-4a39-b747-833780a918e0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:27.824259 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:27.830239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baad3c2-7ec7-4a39-b747-833780a918e0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:27.890624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598747ae-953f-4824-8fcc-9af494ca61fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:27.891333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598747ae-953f-4824-8fcc-9af494ca61fe admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:27.891621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598747ae-953f-4824-8fcc-9af494ca61fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:27.891910 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598747ae-953f-4824-8fcc-9af494ca61fe admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:27.892642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598747ae-953f-4824-8fcc-9af494ca61fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:27.894594 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598747ae-953f-4824-8fcc-9af494ca61fe admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:27.962586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1401/2825] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:47:27 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:27.966420 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b440c1-497d-450e-8827-dbf7fceae5dd admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:27.977695 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b440c1-497d-450e-8827-dbf7fceae5dd admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:27.982543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a40f-91b4-4b94-b212-32dfd592667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:27.983645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a40f-91b4-4b94-b212-32dfd592667b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:27.983645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a40f-91b4-4b94-b212-32dfd592667b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:27.983889 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a40f-91b4-4b94-b212-32dfd592667b admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:27.984644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821a40f-91b4-4b94-b212-32dfd592667b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:27.986266 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821a40f-91b4-4b94-b212-32dfd592667b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:27.996864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1425/2826] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:28.000906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598747ae-953f-4824-8fcc-9af494ca61fe admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:28.011451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598747ae-953f-4824-8fcc-9af494ca61fe admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:28.016810 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57ff72-5273-4ef1-8ce3-18394b7924e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:28.017799 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57ff72-5273-4ef1-8ce3-18394b7924e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:28.018082 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57ff72-5273-4ef1-8ce3-18394b7924e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:28.018357 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57ff72-5273-4ef1-8ce3-18394b7924e6 admin admin] PATH_INFO: `/v3/projects/221dd1a7a810457080f72476b7c3818c/users/a2bc0ed4719c470aa42ad9966b921379/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:28.035950 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57ff72-5273-4ef1-8ce3-18394b7924e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221dd1a7a810457080f72476b7c3818c, user_id=a2bc0ed4719c470aa42ad9966b921379, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:28.038654 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57ff72-5273-4ef1-8ce3-18394b7924e6 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:28.093208 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1402/2827] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:27 2022] PUT /identity/v3/projects/221dd1a7a810457080f72476b7c3818c/users/a2bc0ed4719c470aa42ad9966b921379/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:28.097738 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a40f-91b4-4b94-b212-32dfd592667b admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:28.108236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a40f-91b4-4b94-b212-32dfd592667b admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:28.164338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08a7a4-edd0-4815-938c-7b081a29bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:28.165072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08a7a4-edd0-4815-938c-7b081a29bb20 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:28.165363 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08a7a4-edd0-4815-938c-7b081a29bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:28.165645 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08a7a4-edd0-4815-938c-7b081a29bb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:28.166377 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08a7a4-edd0-4815-938c-7b081a29bb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:28.167976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08a7a4-edd0-4815-938c-7b081a29bb20 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:28.177924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1426/2828] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:28.182085 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57ff72-5273-4ef1-8ce3-18394b7924e6 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:28.193897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57ff72-5273-4ef1-8ce3-18394b7924e6 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:28.197396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:28.198284 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:28.198570 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:28.199038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] PATH_INFO: `/v3/projects/221dd1a7a810457080f72476b7c3818c/users/a2bc0ed4719c470aa42ad9966b921379/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:28.201195 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221dd1a7a810457080f72476b7c3818c, user_id=a2bc0ed4719c470aa42ad9966b921379, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:28.204411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:28.215165 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1403/2829] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:28 2022] PUT /identity/v3/projects/221dd1a7a810457080f72476b7c3818c/users/a2bc0ed4719c470aa42ad9966b921379/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:47:28.226046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7f9d-9abd-4362-8fe5-a8d4b0734704 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:28.226346 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7f9d-9abd-4362-8fe5-a8d4b0734704 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:28.226655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7f9d-9abd-4362-8fe5-a8d4b0734704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:28.253993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-c9bd7f9d-9abd-4362-8fe5-a8d4b0734704 None None] MFA Rules not processed for user `a2bc0ed4719c470aa42ad9966b921379`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:28.257252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-c9bd7f9d-9abd-4362-8fe5-a8d4b0734704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:28.322144 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:28.327891 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1427/2830] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:28.339675 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:28.341534 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:28.400023 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58aaca-8d2e-4896-934b-d0ae9cbe9e78 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:28.403665 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24890df6-a62f-4541-a71d-6f404d6472d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:28.404315 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24890df6-a62f-4541-a71d-6f404d6472d3 service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:28.404623 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24890df6-a62f-4541-a71d-6f404d6472d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:28.404899 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24890df6-a62f-4541-a71d-6f404d6472d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:28.414715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24890df6-a62f-4541-a71d-6f404d6472d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:28.416187 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24890df6-a62f-4541-a71d-6f404d6472d3 service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:28.427018 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1404/2831] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:28.564327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd7f9d-9abd-4362-8fe5-a8d4b0734704 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:28.574280 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd7f9d-9abd-4362-8fe5-a8d4b0734704 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:28.579934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e5c92-c4d0-4d5d-bfa7-2a1fa3226f3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:28.579934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e5c92-c4d0-4d5d-bfa7-2a1fa3226f3f service cinder] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:28.579934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e5c92-c4d0-4d5d-bfa7-2a1fa3226f3f service cinder] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:28.579934 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e5c92-c4d0-4d5d-bfa7-2a1fa3226f3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:28.639398 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e5c92-c4d0-4d5d-bfa7-2a1fa3226f3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:28.641072 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e5c92-c4d0-4d5d-bfa7-2a1fa3226f3f service cinder] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:28.650200 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1428/2832] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:29.130986 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24890df6-a62f-4541-a71d-6f404d6472d3 service cinder] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:29.189168 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:29.193586 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24890df6-a62f-4541-a71d-6f404d6472d3 service cinder] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:29.198064 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:29.198944 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:29.199228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:29.199506 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] PATH_INFO: `/v3/users/ad2e5711551e49c3862a2c7d76e98f71` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:29.201117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2e5711551e49c3862a2c7d76e98f71)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:29.203631 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:29.392200 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2e5711551e49c3862a2c7d76e98f71'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:29.399132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2e5711551e49c3862a2c7d76e98f71'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:29.399856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2e5711551e49c3862a2c7d76e98f71'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:29.400471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2e5711551e49c3862a2c7d76e98f71'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:29.470927 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2e5711551e49c3862a2c7d76e98f71'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:29.489750 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1405/2833] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:47:29 2022] DELETE /identity/v3/users/ad2e5711551e49c3862a2c7d76e98f71 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:29.495864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02339c68-c0dc-452c-82c7-52ee9f49e0fc None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:29.495864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02339c68-c0dc-452c-82c7-52ee9f49e0fc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:29.495864 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02339c68-c0dc-452c-82c7-52ee9f49e0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:29.512682 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-02339c68-c0dc-452c-82c7-52ee9f49e0fc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:29.514057 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-02339c68-c0dc-452c-82c7-52ee9f49e0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:29.582619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1429/2834] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:29.596094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:29.599969 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:29.662990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41282514-04fd-41db-87d3-347bdd568403 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:29.668813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:29.668813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:29.668813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:29.668813 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:29.678442 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:29.679922 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:29.689765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1406/2835] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:29.763499 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02339c68-c0dc-452c-82c7-52ee9f49e0fc None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:29.764976 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-02339c68-c0dc-452c-82c7-52ee9f49e0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:29.879019 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02339c68-c0dc-452c-82c7-52ee9f49e0fc None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:29.884901 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:29.885561 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:29.885849 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:29.886129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:29.887984 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:29.902327 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:29.904021 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:29.911869 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=100 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:29.916772 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1430/2836] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:47:30.146278 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:30.148296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:30.166136 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8bbbe-2b87-40c4-91ca-29405b582ca8 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:30.172235 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:30.174973 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:30.175335 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:30.176346 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] PATH_INFO: `/v3/projects/3003d2da94f148629701a517a70f35ec` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:30.179154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3003d2da94f148629701a517a70f35ec)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:30.182948 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:30.293490 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3003d2da94f148629701a517a70f35ec', 'request_id': 'req-d1e2808b-5210-4992-849f-99302f383ce7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.355931 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3003d2da94f148629701a517a70f35ec', 'request_id': 'req-d1e2808b-5210-4992-849f-99302f383ce7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.363398 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3003d2da94f148629701a517a70f35ec', 'request_id': 'req-d1e2808b-5210-4992-849f-99302f383ce7'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.377755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] The token cache is being invalidate because project 3003d2da94f148629701a517a70f35ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:47:30.378131 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.378871 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.380618 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1407/2837] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:47:30 2022] DELETE /identity/v3/projects/3003d2da94f148629701a517a70f35ec => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:30.384447 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:30.395757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d1276-59d2-4a89-988c-5cbb1fdbe6b4 service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:30.470424 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:30.471482 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:30.471771 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:30.472048 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] PATH_INFO: `/v3/users/a2bc0ed4719c470aa42ad9966b921379` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:30.473334 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2bc0ed4719c470aa42ad9966b921379)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:30.475892 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:30.723252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2bc0ed4719c470aa42ad9966b921379'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.735028 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2bc0ed4719c470aa42ad9966b921379'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.741756 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2bc0ed4719c470aa42ad9966b921379'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.742439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2bc0ed4719c470aa42ad9966b921379'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.743095 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-29b84c5f-bd5a-45bd-835e-80452c6d54a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2bc0ed4719c470aa42ad9966b921379'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:30.819843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1431/2838] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:47:30 2022] DELETE /identity/v3/users/a2bc0ed4719c470aa42ad9966b921379 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:47:31.209152 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:31.210601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:31.219473 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:31.225266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2808b-5210-4992-849f-99302f383ce7 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:31.281179 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:31.282709 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.283217 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.283850 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] PATH_INFO: `/v3/projects/221dd1a7a810457080f72476b7c3818c` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.285155 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=221dd1a7a810457080f72476b7c3818c)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:31.287143 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:31.371755 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221dd1a7a810457080f72476b7c3818c', 'request_id': 'req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:31.418095 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221dd1a7a810457080f72476b7c3818c', 'request_id': 'req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:31.423808 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221dd1a7a810457080f72476b7c3818c', 'request_id': 'req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:31.434859 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] The token cache is being invalidate because project 221dd1a7a810457080f72476b7c3818c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:47:31.435144 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:31.435758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-e0c296f3-9cee-43df-b4bc-bf44b5052fa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:47:31.437417 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1408/2839] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:47:31 2022] DELETE /identity/v3/projects/221dd1a7a810457080f72476b7c3818c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:47:31.442734 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892e89-693c-47b6-bbc7-8f94e2501cd0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.443025 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892e89-693c-47b6-bbc7-8f94e2501cd0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.443302 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892e89-693c-47b6-bbc7-8f94e2501cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.461715 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-91892e89-693c-47b6-bbc7-8f94e2501cd0 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:31.462907 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-91892e89-693c-47b6-bbc7-8f94e2501cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:31.523436 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1432/2840] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:31.531460 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0f3dd-7435-4443-bf7b-4990f7b63747 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.531752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0f3dd-7435-4443-bf7b-4990f7b63747 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.532038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0f3dd-7435-4443-bf7b-4990f7b63747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.550308 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-63b0f3dd-7435-4443-bf7b-4990f7b63747 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:31.551443 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-63b0f3dd-7435-4443-bf7b-4990f7b63747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:31.560175 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1409/2841] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:31.564930 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91892e89-693c-47b6-bbc7-8f94e2501cd0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:31.574832 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91892e89-693c-47b6-bbc7-8f94e2501cd0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:31.580344 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:31.581042 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.581330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.581613 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] PATH_INFO: `/v3/domains` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.582345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:47:31.582679 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:31.584328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:31.597439 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1433/2842] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:47:31.600639 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0f3dd-7435-4443-bf7b-4990f7b63747 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:31.610104 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0f3dd-7435-4443-bf7b-4990f7b63747 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:31.614781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1bf7a-b356-4f07-8ef5-36bf2aa45ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:31.615425 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1bf7a-b356-4f07-8ef5-36bf2aa45ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.615711 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1bf7a-b356-4f07-8ef5-36bf2aa45ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.615990 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1bf7a-b356-4f07-8ef5-36bf2aa45ec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.616746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1bf7a-b356-4f07-8ef5-36bf2aa45ec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:31.618625 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1bf7a-b356-4f07-8ef5-36bf2aa45ec2 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:31.694666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1410/2843] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:47:31 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:47:31.698663 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:31.704136 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:31.709440 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e642be-221d-4e40-8843-0263f2c0d879 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:31.765181 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c39e5-d2b4-4711-97d8-040229c1ef5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:31.765843 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c39e5-d2b4-4711-97d8-040229c1ef5c admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.766123 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c39e5-d2b4-4711-97d8-040229c1ef5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.766397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c39e5-d2b4-4711-97d8-040229c1ef5c admin admin] PATH_INFO: `/v3/users` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.767073 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c39e5-d2b4-4711-97d8-040229c1ef5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:31.769002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c39e5-d2b4-4711-97d8-040229c1ef5c admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:31.875536 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1434/2844] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:47:31 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:47:31.880129 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1bf7a-b356-4f07-8ef5-36bf2aa45ec2 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:31.890676 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1bf7a-b356-4f07-8ef5-36bf2aa45ec2 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:31.895359 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b2177-cf46-4984-80a7-0b7c22b470d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:31.896010 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2177-cf46-4984-80a7-0b7c22b470d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.896296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2177-cf46-4984-80a7-0b7c22b470d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.896604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2177-cf46-4984-80a7-0b7c22b470d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.897487 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b2177-cf46-4984-80a7-0b7c22b470d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:31.899067 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b2177-cf46-4984-80a7-0b7c22b470d5 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:31.909331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1411/2845] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:47:31.913244 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c39e5-d2b4-4711-97d8-040229c1ef5c admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:31.923382 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c39e5-d2b4-4711-97d8-040229c1ef5c admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:31.927942 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c440071-5ed4-4731-8003-c0bb43f2c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:31.928621 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440071-5ed4-4731-8003-c0bb43f2c0ab admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.928908 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440071-5ed4-4731-8003-c0bb43f2c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.929187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440071-5ed4-4731-8003-c0bb43f2c0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.929909 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c440071-5ed4-4731-8003-c0bb43f2c0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:31.931623 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c440071-5ed4-4731-8003-c0bb43f2c0ab admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:31.942811 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1435/2846] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:47:31.946725 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b2177-cf46-4984-80a7-0b7c22b470d5 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:31.957020 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b2177-cf46-4984-80a7-0b7c22b470d5 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:31.961864 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:31.962728 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:31.963013 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:31.963293 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] PATH_INFO: `/v3/projects/a32b5211fe814671baad0e5816e4b949/users/372c815f955b4645bcc2c3f3ed4eb828/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:31.977809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a32b5211fe814671baad0e5816e4b949, user_id=372c815f955b4645bcc2c3f3ed4eb828, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:31.980474 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:32.070471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1412/2847] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:47:31 2022] PUT /identity/v3/projects/a32b5211fe814671baad0e5816e4b949/users/372c815f955b4645bcc2c3f3ed4eb828/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:47:32.075739 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cad256-12c6-406d-8fc6-d291541523dc None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:32.076046 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cad256-12c6-406d-8fc6-d291541523dc None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:32.076331 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cad256-12c6-406d-8fc6-d291541523dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:32.093512 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b0cad256-12c6-406d-8fc6-d291541523dc None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:32.095214 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b0cad256-12c6-406d-8fc6-d291541523dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:32.151638 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1436/2848] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:32.161581 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:32.163343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:32.195888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:32.221258 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df309ad-8e6a-4a4f-a3ba-eda96b8a8ac8 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:32.225589 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:32.226238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:32.226522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:32.226801 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:32.236294 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:32.237735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:32.247084 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1413/2849] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:47:33.432634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e09db-eb32-4021-afb8-a1c326336763 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:33.433207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e09db-eb32-4021-afb8-a1c326336763 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:33.433207 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e09db-eb32-4021-afb8-a1c326336763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:33.459827 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-d88e09db-eb32-4021-afb8-a1c326336763 None None] MFA Rules not processed for user `372c815f955b4645bcc2c3f3ed4eb828`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:33.461081 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-d88e09db-eb32-4021-afb8-a1c326336763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:33.525881 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1437/2850] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:33.536670 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:33.538128 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:33.596339 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346de65-7d4f-4189-a57b-8e9f79ce1555 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:33.600027 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f1f1f-ea5c-4f42-b071-9c55750fc70b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff82e4b1a4248fcbb3566bb7bc84d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:33.600699 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f1f1f-ea5c-4f42-b071-9c55750fc70b service cinder] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:33.600987 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f1f1f-ea5c-4f42-b071-9c55750fc70b service cinder] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:33.601269 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f1f1f-ea5c-4f42-b071-9c55750fc70b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:33.609664 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f1f1f-ea5c-4f42-b071-9c55750fc70b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:33.611758 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f1f1f-ea5c-4f42-b071-9c55750fc70b service cinder] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:33.620424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1414/2851] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:47:35.301764 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aedb5-c5bb-4a98-afc8-aea75d20648d None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:35.302269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aedb5-c5bb-4a98-afc8-aea75d20648d None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:35.302381 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aedb5-c5bb-4a98-afc8-aea75d20648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:35.319304 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-425aedb5-c5bb-4a98-afc8-aea75d20648d None None] MFA Rules not processed for user `372c815f955b4645bcc2c3f3ed4eb828`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:35.320489 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-425aedb5-c5bb-4a98-afc8-aea75d20648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:35.327722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1438/2852] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:43.616463 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=39 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:45.314451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:47:53.683991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a707b-a089-4358-bd03-19d8fc7250b4 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:53.684492 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a707b-a089-4358-bd03-19d8fc7250b4 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:53.684634 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a707b-a089-4358-bd03-19d8fc7250b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:53.701852 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-4d3a707b-a089-4358-bd03-19d8fc7250b4 None None] MFA Rules not processed for user `372c815f955b4645bcc2c3f3ed4eb828`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:47:53.703089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-4d3a707b-a089-4358-bd03-19d8fc7250b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:53.711266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1415/2853] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:47:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:47:53.939355 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aedb5-c5bb-4a98-afc8-aea75d20648d None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:47:53.940781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-425aedb5-c5bb-4a98-afc8-aea75d20648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:53.952856 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aedb5-c5bb-4a98-afc8-aea75d20648d None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:47:53.956406 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:47:53.957068 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:47:53.957347 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:47:53.957618 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:47:53.959882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:47:53.969952 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:47:53.971397 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:47:53.980768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1439/2854] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:47:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:48:03.696663 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=3 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:03.976757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=6 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:16.050227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa2511-1168-4a28-8dbf-b7180c1f461e None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:16.050702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa2511-1168-4a28-8dbf-b7180c1f461e None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:16.050797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa2511-1168-4a28-8dbf-b7180c1f461e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:16.108641 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-34aa2511-1168-4a28-8dbf-b7180c1f461e None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:48:16.109840 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-34aa2511-1168-4a28-8dbf-b7180c1f461e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:16.172451 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1416/2855] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jun 16 00:48:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:48:16.309145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:16.310566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:16.401450 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38084c-407b-489b-9db4-ea1d041a42ff service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:16.406451 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a6858-708b-44fc-911f-b2d1eab282a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:16.407116 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a6858-708b-44fc-911f-b2d1eab282a8 service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:16.407400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a6858-708b-44fc-911f-b2d1eab282a8 service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:16.407674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a6858-708b-44fc-911f-b2d1eab282a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:16.415659 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a6858-708b-44fc-911f-b2d1eab282a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:16.417257 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a6858-708b-44fc-911f-b2d1eab282a8 service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:16.425184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1440/2856] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:48:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:48:26.169718 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=44 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:26.422285 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=41 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:33.461297 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3791347-3470-478c-94f3-8ddd9aa3f5da None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:33.461777 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3791347-3470-478c-94f3-8ddd9aa3f5da None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:33.461882 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3791347-3470-478c-94f3-8ddd9aa3f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:33.480411 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-b3791347-3470-478c-94f3-8ddd9aa3f5da None None] MFA Rules not processed for user `396cdfa4dbf54e66919bc3e054c54154`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:48:33.480781 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-b3791347-3470-478c-94f3-8ddd9aa3f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:33.487857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1417/2857] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jun 16 00:48:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:48:33.500223 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a6858-708b-44fc-911f-b2d1eab282a8 service swift] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:33.510823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a6858-708b-44fc-911f-b2d1eab282a8 service swift] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:33.516535 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5979-9039-46ed-8ebe-888f51c75816 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645bbe41756749b98232d0e2eb2e678a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:33.517186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5979-9039-46ed-8ebe-888f51c75816 service swift] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:33.517475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5979-9039-46ed-8ebe-888f51c75816 service swift] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:33.517757 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5979-9039-46ed-8ebe-888f51c75816 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:33.525842 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db5979-9039-46ed-8ebe-888f51c75816 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:33.527461 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db5979-9039-46ed-8ebe-888f51c75816 service swift] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:33.535781 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1441/2858] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jun 16 00:48:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:48:36.464983 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3791347-3470-478c-94f3-8ddd9aa3f5da None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:36.476770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3791347-3470-478c-94f3-8ddd9aa3f5da None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:36.481974 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:36.482847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:36.483134 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:36.483416 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] PATH_INFO: `/v3/users/372c815f955b4645bcc2c3f3ed4eb828` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:36.484636 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=372c815f955b4645bcc2c3f3ed4eb828)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:36.486548 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:36.666008 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372c815f955b4645bcc2c3f3ed4eb828'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:36.673445 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372c815f955b4645bcc2c3f3ed4eb828'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:36.674101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372c815f955b4645bcc2c3f3ed4eb828'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:36.674701 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372c815f955b4645bcc2c3f3ed4eb828'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:36.737752 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-64bbbb47-06a6-4879-9749-72033ca40d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372c815f955b4645bcc2c3f3ed4eb828'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:36.754765 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1418/2859] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:48:36 2022] DELETE /identity/v3/users/372c815f955b4645bcc2c3f3ed4eb828 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:48:36.979655 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5979-9039-46ed-8ebe-888f51c75816 service swift] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:36.990912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5979-9039-46ed-8ebe-888f51c75816 service swift] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:37.053066 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:37.053924 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.054208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.054486 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] PATH_INFO: `/v3/projects/a32b5211fe814671baad0e5816e4b949` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.055644 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a32b5211fe814671baad0e5816e4b949)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:37.057667 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:37.156090 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a32b5211fe814671baad0e5816e4b949', 'request_id': 'req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:37.218145 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a32b5211fe814671baad0e5816e4b949', 'request_id': 'req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:37.223957 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a32b5211fe814671baad0e5816e4b949', 'request_id': 'req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:37.235091 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] The token cache is being invalidate because project a32b5211fe814671baad0e5816e4b949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:48:37.235376 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:37.236015 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-ea9e76bd-0238-48aa-ba2d-5ea098d8036e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:48:37.237696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1442/2860] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:48:36 2022] DELETE /identity/v3/projects/a32b5211fe814671baad0e5816e4b949 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:48:37.242674 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9466df-5178-4280-9532-a5739bc82ec3 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.242912 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9466df-5178-4280-9532-a5739bc82ec3 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.243199 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9466df-5178-4280-9532-a5739bc82ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.262413 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-7c9466df-5178-4280-9532-a5739bc82ec3 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:48:37.263610 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-7c9466df-5178-4280-9532-a5739bc82ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:37.321072 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1419/2861] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:48:37.329402 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc590f01-e9bc-464f-95a9-c879a1ade183 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.329696 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc590f01-e9bc-464f-95a9-c879a1ade183 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.329981 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc590f01-e9bc-464f-95a9-c879a1ade183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.347769 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-fc590f01-e9bc-464f-95a9-c879a1ade183 None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:48:37.348987 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-fc590f01-e9bc-464f-95a9-c879a1ade183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:37.356768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1443/2862] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:48:37.360616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9466df-5178-4280-9532-a5739bc82ec3 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:37.370051 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9466df-5178-4280-9532-a5739bc82ec3 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:37.375065 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:37.375708 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.375991 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.376271 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.377038 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:48:37.377347 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:37.378921 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:37.390763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1420/2863] 10.222.0.20 () {58 vars in 1152 bytes} [Thu Jun 16 00:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:48:37.394707 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc590f01-e9bc-464f-95a9-c879a1ade183 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:37.403788 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc590f01-e9bc-464f-95a9-c879a1ade183 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:37.408383 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82411fa0-f4f0-4b46-a442-a26f79dd5ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:37.409060 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82411fa0-f4f0-4b46-a442-a26f79dd5ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.409345 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82411fa0-f4f0-4b46-a442-a26f79dd5ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.409625 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82411fa0-f4f0-4b46-a442-a26f79dd5ce7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.410325 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82411fa0-f4f0-4b46-a442-a26f79dd5ce7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:37.412198 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82411fa0-f4f0-4b46-a442-a26f79dd5ce7 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:37.522506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1444/2864] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:48:37 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:48:37.526603 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:37.537221 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fdd00-0dd2-496e-a819-53e58d919f3d admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:37.584642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:37.594925 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa327-e779-4859-8704-60c657cb8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:37.595608 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa327-e779-4859-8704-60c657cb8b05 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.595897 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa327-e779-4859-8704-60c657cb8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.596177 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa327-e779-4859-8704-60c657cb8b05 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.597036 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafa327-e779-4859-8704-60c657cb8b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:37.598836 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafa327-e779-4859-8704-60c657cb8b05 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:37.677054 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1421/2865] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:48:37 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:48:37.679286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82411fa0-f4f0-4b46-a442-a26f79dd5ce7 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:37.689727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82411fa0-f4f0-4b46-a442-a26f79dd5ce7 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:37.694368 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f416f-ba11-4b09-a23e-f1ee38b1dbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:37.695030 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f416f-ba11-4b09-a23e-f1ee38b1dbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.695318 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f416f-ba11-4b09-a23e-f1ee38b1dbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.695603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f416f-ba11-4b09-a23e-f1ee38b1dbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.696332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f416f-ba11-4b09-a23e-f1ee38b1dbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:37.697965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f416f-ba11-4b09-a23e-f1ee38b1dbf3 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:37.709146 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1445/2866] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:48:37.717256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa327-e779-4859-8704-60c657cb8b05 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:37.727215 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa327-e779-4859-8704-60c657cb8b05 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:37.732114 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0fa73-91fc-40db-8429-43c7dbe0be9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:37.733203 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0fa73-91fc-40db-8429-43c7dbe0be9a admin admin] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.733500 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0fa73-91fc-40db-8429-43c7dbe0be9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.733787 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0fa73-91fc-40db-8429-43c7dbe0be9a admin admin] PATH_INFO: `/v3/roles` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.734522 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0fa73-91fc-40db-8429-43c7dbe0be9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:37.736113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0fa73-91fc-40db-8429-43c7dbe0be9a admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:37.746017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1422/2867] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:48:37.749918 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f416f-ba11-4b09-a23e-f1ee38b1dbf3 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:37.759736 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f416f-ba11-4b09-a23e-f1ee38b1dbf3 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:37.764967 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:37.766816 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] REQUEST_METHOD: `PUT` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.767100 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.767412 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] PATH_INFO: `/v3/projects/5e9d772a2cb14bbbaaa2dcaf5d730f6e/users/d0aeb18780734fecbbd6bc8f9202f84d/roles/64517449a51047a3b2ce1079b4912f57` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.783121 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9d772a2cb14bbbaaa2dcaf5d730f6e, user_id=d0aeb18780734fecbbd6bc8f9202f84d, role_id=64517449a51047a3b2ce1079b4912f57)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:37.785731 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:37.851628 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1446/2868] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:48:37 2022] PUT /identity/v3/projects/5e9d772a2cb14bbbaaa2dcaf5d730f6e/users/d0aeb18780734fecbbd6bc8f9202f84d/roles/64517449a51047a3b2ce1079b4912f57 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:48:37.863742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae19d-8016-421a-aa22-42da2e2db80d None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:37.864060 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae19d-8016-421a-aa22-42da2e2db80d None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:37.864343 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae19d-8016-421a-aa22-42da2e2db80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:37.882340 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-a85ae19d-8016-421a-aa22-42da2e2db80d None None] MFA Rules not processed for user `1aa1627cba2640fdb90f9d237f108268`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:48:37.883520 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a85ae19d-8016-421a-aa22-42da2e2db80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:37.941302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1423/2869] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:48:37.951354 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:37.953332 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:37.967620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:38.009431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205edf12-5385-4e5d-b4ff-d2e612b413ef admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:38.013685 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afacf7d-16d0-49a6-91b3-0d33b059a7ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:38.014330 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afacf7d-16d0-49a6-91b3-0d33b059a7ee service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:38.014619 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afacf7d-16d0-49a6-91b3-0d33b059a7ee service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:38.014900 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afacf7d-16d0-49a6-91b3-0d33b059a7ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:38.024111 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afacf7d-16d0-49a6-91b3-0d33b059a7ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:38.025566 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afacf7d-16d0-49a6-91b3-0d33b059a7ee service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:38.040611 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1447/2870] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:48:43.135599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f0fd-c033-4167-b95c-d429804601f0 None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:43.136295 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f0fd-c033-4167-b95c-d429804601f0 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:43.136698 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f0fd-c033-4167-b95c-d429804601f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:43.166429 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-9fb5f0fd-c033-4167-b95c-d429804601f0 None None] MFA Rules not processed for user `d0aeb18780734fecbbd6bc8f9202f84d`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:48:43.167782 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9fb5f0fd-c033-4167-b95c-d429804601f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:43.237016 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1424/2871] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:48:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:48:43.241640 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afacf7d-16d0-49a6-91b3-0d33b059a7ee service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:43.252032 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afacf7d-16d0-49a6-91b3-0d33b059a7ee service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:43.256895 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdb2b4-0e01-4094-92a8-a4ce069621e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:43.257570 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdb2b4-0e01-4094-92a8-a4ce069621e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:43.257855 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdb2b4-0e01-4094-92a8-a4ce069621e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:43.258129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdb2b4-0e01-4094-92a8-a4ce069621e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:43.258823 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdb2b4-0e01-4094-92a8-a4ce069621e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:43.260759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdb2b4-0e01-4094-92a8-a4ce069621e8 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:43.370235 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1448/2872] 10.222.0.20 () {60 vars in 1152 bytes} [Thu Jun 16 00:48:43 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:48:43.373348 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f0fd-c033-4167-b95c-d429804601f0 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:43.384847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f0fd-c033-4167-b95c-d429804601f0 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:43.442682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f61474-3e7b-4aec-8fad-2ec90ff8f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:43.443396 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f61474-3e7b-4aec-8fad-2ec90ff8f5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:43.443682 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f61474-3e7b-4aec-8fad-2ec90ff8f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:43.443962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f61474-3e7b-4aec-8fad-2ec90ff8f5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:43.444692 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f61474-3e7b-4aec-8fad-2ec90ff8f5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:43.446615 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f61474-3e7b-4aec-8fad-2ec90ff8f5c0 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:43.455563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:43.544907 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1425/2873] 10.222.0.20 () {60 vars in 1143 bytes} [Thu Jun 16 00:48:43 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:48:43.549657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdb2b4-0e01-4094-92a8-a4ce069621e8 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:43.560923 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdb2b4-0e01-4094-92a8-a4ce069621e8 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:43.565751 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0750c-7434-4ab6-b68a-8e1a59c8ca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:43.566518 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0750c-7434-4ab6-b68a-8e1a59c8ca63 admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:43.566759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0750c-7434-4ab6-b68a-8e1a59c8ca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:43.567036 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0750c-7434-4ab6-b68a-8e1a59c8ca63 admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:43.567777 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0750c-7434-4ab6-b68a-8e1a59c8ca63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:43.569400 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0750c-7434-4ab6-b68a-8e1a59c8ca63 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:43.581431 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1449/2874] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:48:43.585600 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f61474-3e7b-4aec-8fad-2ec90ff8f5c0 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:43.596966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f61474-3e7b-4aec-8fad-2ec90ff8f5c0 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:43.602157 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bec89-d760-4a8b-aa96-1a3dba7b8ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:43.603047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bec89-d760-4a8b-aa96-1a3dba7b8ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:43.603334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bec89-d760-4a8b-aa96-1a3dba7b8ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:43.603616 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bec89-d760-4a8b-aa96-1a3dba7b8ecc admin admin] PATH_INFO: `/v3/projects/b42747645b014584be62f7cf59b63b63/users/f73a86cca469453692b32e14f2182e04/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:43.618332 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bec89-d760-4a8b-aa96-1a3dba7b8ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42747645b014584be62f7cf59b63b63, user_id=f73a86cca469453692b32e14f2182e04, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:43.621014 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bec89-d760-4a8b-aa96-1a3dba7b8ecc admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:43.681917 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1426/2875] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:48:43 2022] PUT /identity/v3/projects/b42747645b014584be62f7cf59b63b63/users/f73a86cca469453692b32e14f2182e04/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:48:43.686392 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0750c-7434-4ab6-b68a-8e1a59c8ca63 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:43.696741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0750c-7434-4ab6-b68a-8e1a59c8ca63 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:43.753390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a48d0c-edf4-4add-a21e-f5468e9a8a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:43.754086 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a48d0c-edf4-4add-a21e-f5468e9a8a6b admin admin] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:43.754378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a48d0c-edf4-4add-a21e-f5468e9a8a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:43.754661 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a48d0c-edf4-4add-a21e-f5468e9a8a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:43.755390 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a48d0c-edf4-4add-a21e-f5468e9a8a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:43.757037 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a48d0c-edf4-4add-a21e-f5468e9a8a6b admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:43.766724 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1450/2876] 10.222.0.20 () {58 vars in 1121 bytes} [Thu Jun 16 00:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:48:43.770933 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bec89-d760-4a8b-aa96-1a3dba7b8ecc admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:43.782026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bec89-d760-4a8b-aa96-1a3dba7b8ecc admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:43.786687 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:43.787564 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] REQUEST_METHOD: `PUT` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:43.787847 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:43.789784 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] PATH_INFO: `/v3/projects/b42747645b014584be62f7cf59b63b63/users/f73a86cca469453692b32e14f2182e04/roles/11d374b9ca394116bbb541aa3ed7cc2d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:43.790188 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42747645b014584be62f7cf59b63b63, user_id=f73a86cca469453692b32e14f2182e04, role_id=11d374b9ca394116bbb541aa3ed7cc2d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:43.792874 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:43.803760 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1427/2877] 10.222.0.20 () {60 vars in 1482 bytes} [Thu Jun 16 00:48:43 2022] PUT /identity/v3/projects/b42747645b014584be62f7cf59b63b63/users/f73a86cca469453692b32e14f2182e04/roles/11d374b9ca394116bbb541aa3ed7cc2d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:48:49.143311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6ecb-ff38-45cf-b0a7-b318ff389cbb None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:49.143870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6ecb-ff38-45cf-b0a7-b318ff389cbb None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:49.143870 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6ecb-ff38-45cf-b0a7-b318ff389cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:49.173182 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-b10a6ecb-ff38-45cf-b0a7-b318ff389cbb None None] MFA Rules not processed for user `f73a86cca469453692b32e14f2182e04`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:48:49.174579 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-b10a6ecb-ff38-45cf-b0a7-b318ff389cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:49.248580 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1451/2878] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:48:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:48:49.265260 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:49.266970 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:49.345302 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a93e4b-f3e1-44b0-abef-d1281bd9a296 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:49.350829 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:49.351427 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:49.351715 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:49.352004 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:49.413154 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:49.414746 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:49.424604 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1428/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:48:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:48:52.829812 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02c47-cb53-41de-97aa-4e47c0ab87e0 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:52.830311 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02c47-cb53-41de-97aa-4e47c0ab87e0 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:52.830410 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02c47-cb53-41de-97aa-4e47c0ab87e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:52.837328 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:52.848322 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-a3c02c47-cb53-41de-97aa-4e47c0ab87e0 None None] MFA Rules not processed for user `d0aeb18780734fecbbd6bc8f9202f84d`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:48:52.849577 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a3c02c47-cb53-41de-97aa-4e47c0ab87e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:52.856456 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1452/2880] 10.222.0.20 () {58 vars in 957 bytes} [Thu Jun 16 00:48:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:48:53.217609 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:48:53.219037 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:53.287621 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:48:53.325292 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ab02e-1d1c-4adc-babe-d822faa16374 service nova] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:48:53.332878 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:48:53.333573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:48:53.333860 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:48:53.334142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:48:53.335966 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:48:53.349447 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:48:53.351223 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:48:53.363567 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1429/2881] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:48:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:49:02.903435 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:49:03.463599 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=18 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:49:13.491564 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c02c47-cb53-41de-97aa-4e47c0ab87e0 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:49:13.493052 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a3c02c47-cb53-41de-97aa-4e47c0ab87e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:13.584538 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c02c47-cb53-41de-97aa-4e47c0ab87e0 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:49:13.588252 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:49:13.595226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:49:13.595226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:49:13.595226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:49:13.595226 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:13.663687 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:49:13.665186 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:49:13.676732 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1453/2882] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:49:17.144716 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:49:17.146142 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:17.159092 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691e3fc-0449-4dfe-98f5-20fdb9cd1688 service placement] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:49:17.164399 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:49:17.165185 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:49:17.165469 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:49:17.165748 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:49:17.167471 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:17.179099 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:49:17.180702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:49:17.192089 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1430/2883] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:49:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:49:20.874204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:49:20.875672 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:20.889543 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7185d6-40bd-4234-944e-125aff382aa7 service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:49:20.897018 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:49:20.898506 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:49:20.898993 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:49:20.899459 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] PATH_INFO: `/v3/users/517928d85b764c089c6dd5b2d088639c` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:49:20.901156 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=517928d85b764c089c6dd5b2d088639c)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:49:20.903109 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:49:21.088286 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517928d85b764c089c6dd5b2d088639c'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.100449 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517928d85b764c089c6dd5b2d088639c'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.107229 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517928d85b764c089c6dd5b2d088639c'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.107938 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517928d85b764c089c6dd5b2d088639c'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.108574 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517928d85b764c089c6dd5b2d088639c'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.206078 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1454/2884] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:49:20 2022] DELETE /identity/v3/users/517928d85b764c089c6dd5b2d088639c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:49:21.516832 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:49:21.527954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd213-dae1-4b9d-9df2-491bdb8c48dd service placement] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:49:21.586219 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:49:21.587015 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:49:21.587296 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:49:21.587574 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] PATH_INFO: `/v3/projects/3a122fcfab7f418baf78d7b742fa20db` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:49:21.588797 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a122fcfab7f418baf78d7b742fa20db)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:49:21.590680 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:49:21.749613 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a122fcfab7f418baf78d7b742fa20db', 'request_id': 'req-5375670f-f6b3-4f02-89cc-97f375236e0e'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.812047 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a122fcfab7f418baf78d7b742fa20db', 'request_id': 'req-5375670f-f6b3-4f02-89cc-97f375236e0e'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.818361 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a122fcfab7f418baf78d7b742fa20db', 'request_id': 'req-5375670f-f6b3-4f02-89cc-97f375236e0e'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.830113 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] The token cache is being invalidate because project 3a122fcfab7f418baf78d7b742fa20db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:49:21.830432 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.831115 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:21.832695 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1431/2885] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:49:21 2022] DELETE /identity/v3/projects/3a122fcfab7f418baf78d7b742fa20db => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:49:21.836822 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:49:21.838164 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:21.852002 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe5eec-d320-4518-847b-1406bbcc70f6 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:49:21.872829 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:49:21.910760 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:49:21.911636 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:49:21.911916 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:49:21.912188 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] PATH_INFO: `/v3/users/dc61a69b9a8248ac8ec5a21c95441852` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:49:21.913391 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc61a69b9a8248ac8ec5a21c95441852)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:49:21.915386 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:49:22.083671 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc61a69b9a8248ac8ec5a21c95441852'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.095547 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc61a69b9a8248ac8ec5a21c95441852'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.103641 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc61a69b9a8248ac8ec5a21c95441852'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.104358 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc61a69b9a8248ac8ec5a21c95441852'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.105099 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc61a69b9a8248ac8ec5a21c95441852'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.178236 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1455/2886] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:49:21 2022] DELETE /identity/v3/users/dc61a69b9a8248ac8ec5a21c95441852 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:49:22.407539 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:49:22.409666 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:22.423624 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375670f-f6b3-4f02-89cc-97f375236e0e admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:49:22.482086 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:49:22.482962 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:49:22.483249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:49:22.483527 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] PATH_INFO: `/v3/projects/462ba7aac9634b0eb45f69eeed5823ec` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:49:22.484742 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] RBAC: Authorizing `identity:delete_project(project_id=462ba7aac9634b0eb45f69eeed5823ec)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:49:22.486642 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:49:22.509535 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:49:22.624867 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '462ba7aac9634b0eb45f69eeed5823ec', 'request_id': 'req-c34eaa41-081e-4881-aa16-651d5cbaecae'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.697620 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '462ba7aac9634b0eb45f69eeed5823ec', 'request_id': 'req-c34eaa41-081e-4881-aa16-651d5cbaecae'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.703770 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '462ba7aac9634b0eb45f69eeed5823ec', 'request_id': 'req-c34eaa41-081e-4881-aa16-651d5cbaecae'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.715162 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] The token cache is being invalidate because project 462ba7aac9634b0eb45f69eeed5823ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:49:22.715454 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.716117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:49:22.717690 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1432/2887] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:49:22 2022] DELETE /identity/v3/projects/462ba7aac9634b0eb45f69eeed5823ec => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:49:26.315719 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:49:26.317208 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:26.377098 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6e12b-4d60-4ca3-b3d1-f862b7d877b9 admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:49:26.427187 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:49:26.427835 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:49:26.428129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:49:26.428413 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:49:26.430247 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:49:26.440728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:49:26.492189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:49:26.493654 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:49:26.502950 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1456/2888] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:49:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:49:32.712022 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:49:36.542076 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=24 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:50:50.908061 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:50:50.909648 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:50:50.922350 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eaa41-081e-4881-aa16-651d5cbaecae admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:50:50.926809 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:50:50.927459 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:50:50.927745 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:50:50.928026 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:50:50.929757 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:50:50.939696 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:50:50.941153 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:50:50.949110 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1433/2889] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:50:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:51:00.945584 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=6 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:51:22.822475 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:51:22.823871 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:51:22.834857 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc63baf-4f20-46c6-b449-eda28577a01e service neutron] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:51:22.839056 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae0e51-6d5e-4d90-8751-82e1ef092275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:51:22.839716 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0e51-6d5e-4d90-8751-82e1ef092275 service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:51:22.839996 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0e51-6d5e-4d90-8751-82e1ef092275 service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:51:22.840271 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0e51-6d5e-4d90-8751-82e1ef092275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:51:22.841983 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-aaae0e51-6d5e-4d90-8751-82e1ef092275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:51:22.914242 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae0e51-6d5e-4d90-8751-82e1ef092275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:51:22.915691 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae0e51-6d5e-4d90-8751-82e1ef092275 service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:51:22.926937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1457/2890] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:51:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:51:24.100958 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:51:24.110338 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79742003-e750-4866-88ac-315f482d0ea1 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:51:24.114101 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7415f-db0b-4368-9708-952ff1221342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:51:24.114747 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7415f-db0b-4368-9708-952ff1221342 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:51:24.115039 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7415f-db0b-4368-9708-952ff1221342 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:51:24.115322 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7415f-db0b-4368-9708-952ff1221342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:51:24.117174 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9ad7415f-db0b-4368-9708-952ff1221342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:51:24.130780 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7415f-db0b-4368-9708-952ff1221342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:51:24.132225 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7415f-db0b-4368-9708-952ff1221342 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:51:24.143228 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1434/2891] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:51:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:51:32.922069 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=34 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:51:34.138256 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=6 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:51:36.529844 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417d3fa-23c2-422f-90c2-1d1579ba8398 None None] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:51:36.530438 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417d3fa-23c2-422f-90c2-1d1579ba8398 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:51:36.530565 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417d3fa-23c2-422f-90c2-1d1579ba8398 None None] PATH_INFO: `/` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:51:36.531848 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1458/2892] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:51:36.537879 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d662a68-cd26-45de-bd46-11315858990b None None] REQUEST_METHOD: `POST` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:51:36.538363 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d662a68-cd26-45de-bd46-11315858990b None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:51:36.538702 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d662a68-cd26-45de-bd46-11315858990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:51:36.580068 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.auth.core [None req-6d662a68-cd26-45de-bd46-11315858990b None None] MFA Rules not processed for user `96df1ee2cbc14bcc8a8a250fd8f349dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:51:36.581327 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-6d662a68-cd26-45de-bd46-11315858990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:51:36.597266 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1435/2893] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:51:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:51:36.609646 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417d3fa-23c2-422f-90c2-1d1579ba8398 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:51:36.621169 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417d3fa-23c2-422f-90c2-1d1579ba8398 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:51:36.624656 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501dac93-ace6-4940-b905-21ab7e5bc56f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:51:36.625315 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dac93-ace6-4940-b905-21ab7e5bc56f service neutron] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:51:36.625601 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dac93-ace6-4940-b905-21ab7e5bc56f service neutron] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:51:36.625882 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dac93-ace6-4940-b905-21ab7e5bc56f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:51:36.632851 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501dac93-ace6-4940-b905-21ab7e5bc56f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:51:36.634275 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501dac93-ace6-4940-b905-21ab7e5bc56f service neutron] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:51:36.642134 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1459/2894] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:51:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:51:36.722279 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5ba12-1176-4028-93bf-c7058c25d976 None None] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:51:36.722573 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5ba12-1176-4028-93bf-c7058c25d976 None None] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:51:36.722857 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5ba12-1176-4028-93bf-c7058c25d976 None None] PATH_INFO: `/` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:51:36.724239 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1436/2895] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jun 16 00:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:51:36.730338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059ae0-4334-4e19-9dd3-fc8cc9927a77 None None] REQUEST_METHOD: `POST` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:51:36.730632 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059ae0-4334-4e19-9dd3-fc8cc9927a77 None None] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:51:36.730913 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059ae0-4334-4e19-9dd3-fc8cc9927a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:51:36.766184 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.auth.core [None req-39059ae0-4334-4e19-9dd3-fc8cc9927a77 None None] MFA Rules not processed for user `dea47e0ec737422c8f679e21174f8cab`. Rule list: `[]` (Enabled: `True`). {{(pid=86686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:51:36.767374 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-39059ae0-4334-4e19-9dd3-fc8cc9927a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:51:36.777161 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1460/2896] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jun 16 00:51:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:51:36.787476 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5ba12-1176-4028-93bf-c7058c25d976 None None] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:51:36.798812 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5ba12-1176-4028-93bf-c7058c25d976 None None] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:51:36.802331 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9c914-66f3-4185-a923-1653d6928272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96df1ee2cbc14bcc8a8a250fd8f349dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:51:36.802954 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9c914-66f3-4185-a923-1653d6928272 service neutron] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:51:36.803237 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9c914-66f3-4185-a923-1653d6928272 service neutron] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:51:36.803513 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9c914-66f3-4185-a923-1653d6928272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:51:36.813334 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9c914-66f3-4185-a923-1653d6928272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:51:36.814735 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9c914-66f3-4185-a923-1653d6928272 service neutron] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:51:36.825146 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1437/2897] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:51:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:51:46.763676 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=15 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:51:46.820591 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=15 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:53:04.763728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39059ae0-4334-4e19-9dd3-fc8cc9927a77 None None] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:53:04.763728 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-39059ae0-4334-4e19-9dd3-fc8cc9927a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:04.776684 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39059ae0-4334-4e19-9dd3-fc8cc9927a77 None None] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:53:04.782937 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:53:04.783603 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:53:04.783893 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:53:04.784585 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:53:04.786306 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:04.800634 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:53:04.802809 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:53:04.815674 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1461/2898] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:53:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:53:07.306731 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9c914-66f3-4185-a923-1653d6928272 service neutron] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:53:07.318479 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9c914-66f3-4185-a923-1653d6928272 service neutron] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:53:07.323906 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea47e0ec737422c8f679e21174f8cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:53:07.324601 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:53:07.324964 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:53:07.325249 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:53:07.326998 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:07.339597 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:53:07.341209 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:53:07.352373 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1438/2899] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:53:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:53:13.403284 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:53:13.405204 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:13.486107 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154933bb-7ca9-4cb7-a83b-477baeee1705 service nova] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:53:13.491129 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:53:13.491774 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] REQUEST_METHOD: `GET` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:53:13.492061 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:53:13.492338 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:53:13.494139 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:13.506117 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:53:13.507709 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:53:13.518523 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1462/2900] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:53:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:53:17.053504 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:53:17.054227 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:17.121763 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e9f03-7312-4efd-83f1-e8c54930624c service nova] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:53:17.126273 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:53:17.127132 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:53:17.128094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:53:17.128094 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] PATH_INFO: `/v3/users/d0aeb18780734fecbbd6bc8f9202f84d` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:53:17.129029 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0aeb18780734fecbbd6bc8f9202f84d)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:53:17.130965 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:53:17.320268 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0aeb18780734fecbbd6bc8f9202f84d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.327077 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0aeb18780734fecbbd6bc8f9202f84d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.327736 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0aeb18780734fecbbd6bc8f9202f84d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.328325 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0aeb18780734fecbbd6bc8f9202f84d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.387530 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0aeb18780734fecbbd6bc8f9202f84d'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.405333 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1439/2901] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:53:17 2022] DELETE /identity/v3/users/d0aeb18780734fecbbd6bc8f9202f84d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:53:17.658389 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:53:17.674657 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdf3d8-5dfd-49e3-8d64-6065b65acf7a service placement] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:53:17.731422 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:53:17.732269 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:53:17.732582 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:53:17.732863 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] PATH_INFO: `/v3/projects/5e9d772a2cb14bbbaaa2dcaf5d730f6e` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:53:17.734020 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9d772a2cb14bbbaaa2dcaf5d730f6e)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:53:17.735965 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:53:17.841128 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9d772a2cb14bbbaaa2dcaf5d730f6e', 'request_id': 'req-1890858d-5ce6-42aa-9923-f9f1583709cc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.908768 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9d772a2cb14bbbaaa2dcaf5d730f6e', 'request_id': 'req-1890858d-5ce6-42aa-9923-f9f1583709cc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.914516 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9d772a2cb14bbbaaa2dcaf5d730f6e', 'request_id': 'req-1890858d-5ce6-42aa-9923-f9f1583709cc'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.925462 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] The token cache is being invalidate because project 5e9d772a2cb14bbbaaa2dcaf5d730f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:53:17.925749 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.926378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:17.927906 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1463/2902] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:53:17 2022] DELETE /identity/v3/projects/5e9d772a2cb14bbbaaa2dcaf5d730f6e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:53:17.931977 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:53:17.933419 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:17.946930 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936b239-983b-43f5-adad-6658cbd5cefe admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:53:18.006017 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:53:18.006888 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:53:18.007171 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:53:18.007446 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] PATH_INFO: `/v3/users/f73a86cca469453692b32e14f2182e04` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:53:18.008697 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f73a86cca469453692b32e14f2182e04)` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:53:18.010598 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:53:18.016087 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:53:18.249117 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f73a86cca469453692b32e14f2182e04'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.257270 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f73a86cca469453692b32e14f2182e04'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.258034 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f73a86cca469453692b32e14f2182e04'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.258563 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f73a86cca469453692b32e14f2182e04'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.331461 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.notifications [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f73a86cca469453692b32e14f2182e04'} {{(pid=86685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.349856 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1440/2903] 10.222.0.20 () {58 vars in 1223 bytes} [Thu Jun 16 00:53:17 2022] DELETE /identity/v3/users/f73a86cca469453692b32e14f2182e04 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:53:18.584276 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] Authenticating user token {{(pid=86686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:53:18.585741 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.fernet_utils [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:18.601059 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890858d-5ce6-42aa-9923-f9f1583709cc admin admin] Validating token access rules against request {{(pid=86686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:53:18.604944 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:53:18.659722 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1aa1627cba2640fdb90f9d237f108268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e616646e114f6f885b0d1df925801d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:53:18.660620 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:53:18.660912 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:53:18.661189 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] PATH_INFO: `/v3/projects/b42747645b014584be62f7cf59b63b63` {{(pid=86686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:53:18.662349 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42747645b014584be62f7cf59b63b63)` {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:53:18.664234 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] RBAC: Authorization granted {{(pid=86686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:53:18.794759 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42747645b014584be62f7cf59b63b63', 'request_id': 'req-57fddb50-fd74-422f-9b9c-084730e925d5'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.890961 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42747645b014584be62f7cf59b63b63', 'request_id': 'req-57fddb50-fd74-422f-9b9c-084730e925d5'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.896717 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42747645b014584be62f7cf59b63b63', 'request_id': 'req-57fddb50-fd74-422f-9b9c-084730e925d5'} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.908291 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] The token cache is being invalidate because project b42747645b014584be62f7cf59b63b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:53:18.908615 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.909378 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG keystone.notifications [None req-57fddb50-fd74-422f-9b9c-084730e925d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:53:18.910928 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: [pid: 86686|app: 0|req: 1464/2904] 10.222.0.20 () {58 vars in 1232 bytes} [Thu Jun 16 00:53:18 2022] DELETE /identity/v3/projects/b42747645b014584be62f7cf59b63b63 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:53:28.346512 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:53:28.904727 dv-cinder-803823-11-iscsi devstack@keystone.service[86686]: DEBUG dbcounter [-] [86686] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:53:59.180238 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] Authenticating user token {{(pid=86685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:53:59.181684 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:59.250369 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3e59-521c-49e0-98d1-cb9f5f326553 admin admin] Validating token access rules against request {{(pid=86685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:53:59.310206 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d86e-6521-47b8-ae05-037b878b6c69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc54fd2a9dd44b6b658c44edf170e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c36bfe335ce4f4fb3a1ce620eda03af', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:53:59.310861 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d86e-6521-47b8-ae05-037b878b6c69 service placement] REQUEST_METHOD: `GET` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:53:59.311145 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d86e-6521-47b8-ae05-037b878b6c69 service placement] SCRIPT_NAME: `/identity` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:53:59.311424 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d86e-6521-47b8-ae05-037b878b6c69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:53:59.313214 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.fernet_utils [None req-5f68d86e-6521-47b8-ae05-037b878b6c69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:53:59.324886 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d86e-6521-47b8-ae05-037b878b6c69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:53:59.326472 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d86e-6521-47b8-ae05-037b878b6c69 service placement] RBAC: Authorization granted {{(pid=86685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:53:59.338894 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: [pid: 86685|app: 0|req: 1441/2905] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jun 16 00:53:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:54:09.333657 dv-cinder-803823-11-iscsi devstack@keystone.service[86685]: DEBUG dbcounter [-] [86685] Writing DB stats keystone:SELECT=58 {{(pid=86685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}